Assessment Solution Strategist

Pluralsight

1d • Hybrid

About The Position

An Assessment Solution Strategist is responsible for the design, development, continuous health and management of all components of Pluralsight assessments. This role involves creation of assessment blueprints, prompt generation for item creation, data and item analysis, forms management, continuous quality improvement and some vendor relationship and project management. We are seeking a detail-oriented Assessment Solution Strategist to join our team in creating high-quality assessments that accurately measure skills, knowledge, and competencies. You will craft valid, reliable, and engaging assessments that drive learning and performance outcomes. Whether it's building learning checks, certification prep, or high-integrity, criterion-referenced assessments you’ll play a key part in ensuring learners are accurately and fairly evaluated across a variety of technical and professional domains. A successful Assessment Solution Strategist will have knowledge of and experience in executing the full life cycle of psychometrically grounded design, development and quality assurance across the entire assessment lifecycle. Who you’re committed to being: You enjoy learning and are open to new ways of doing things. You are not afraid to be yourself, experiment, make mistakes and learn from them, ask questions, or voice your concerns. When communicating you are self-aware, insightful, and proactive. You are a team member first and individual contributor second. You are aware that high-performing teams are only as strong as their weakest link. You believe in continuous improvement and request frequent feedback from others. What you’ll do: Design assessment blueprints and related assessment deliverables. Quality assessment performance management: item/test analysis, calibration, equating/scaling, reliability/validity checks. Data analysis and ongoing assessment monitoring and continuous quality improvement Ensuring that assessment content aligns with curriculum standards, learning objectives, and educational goals. Leverage AI to optimize processes and deliverables. Establish guidelines and standards for creating assessment content to maintain consistency and reliability. Staying updated on best practices in assessment design and research to inform assessment development. Documentation of processes, procedures, and decisions related to assessment design, development and continuous quality monitoring and improvement Design and develop valid and reliable assessments that align with learning objectives and competency frameworks. Monitors assessment portfolio across the assessment lifecycle Manage SOWs for vendor relations Collaborates with internal and external partners throughout the design, development, and deployment of assessments. Workload management, ability to handle multiple tasks and pay attention to detail Individual workload project management Staying current with assessment trends, learning science, and industry standards (e.g., AERA, APA, NCME). Support the development of digital assets or other collateral as related to assessment. As required, provides training, consultation, or guidance to team members and partners on assessment models, assessment design and development practices, and data-driven, evidence-based practices in assessment. Performs other related duties as assigned. An appetite for curiosity and rapid innovation.

Requirements

• Bachelor’s degree in academic, psychological or educational measurement, instructional design, psychometrics, or a related field.
• 3+ years of relevant professional designing and developing high integrity assessments or background in educational measurement.
• Strong command of assessment types, taxonomy, and principles of validity and reliability.
• Excellent writing, editing, and quality assurance skills.
• Experience working with vendors, SMEs and cross-functional teams.
• Experience learning management systems (LMSs), authoring tools, and item banks.
• Innovation driven, growth mindset.
• Candidates should have strong organizational and project management skills to effectively manage multiple assessment development projects simultaneously, meet deadlines, and coordinate with team members.
• Strong analytical skills are important for analyzing assessment data, identifying trends, and making data-driven decisions to improve assessment content and processes.
• Attention to detail is crucial for ensuring the accuracy, clarity, and consistency of assessment content.
• Previous experience in assessment development, instructional design, curriculum development measurement, or a related field is required.
• Masters Degree or 3-5 years experience in assessment design, development or measurement.

Nice To Haves

• Experience with technical/IT content or certifications (e.g., AWS, Azure, Python, security).
• Background in psychometrics or data analysis using tools like Excel, R, or SPSS.
• Knowledge of accessibility and universal design for learning (UDL) standards.
• Experience with adaptive testing or automated assessment tools.
• Familiarity with Pluralsight platform and value proposition
• Experience with SaaS software (eg. ZenDesk, Asana, Confluence, and Smartsheet)

Responsibilities

• Design assessment blueprints and related assessment deliverables.
• Quality assessment performance management: item/test analysis, calibration, equating/scaling, reliability/validity checks.
• Data analysis and ongoing assessment monitoring and continuous quality improvement
• Ensuring that assessment content aligns with curriculum standards, learning objectives, and educational goals.
• Leverage AI to optimize processes and deliverables.
• Establish guidelines and standards for creating assessment content to maintain consistency and reliability.
• Staying updated on best practices in assessment design and research to inform assessment development.
• Documentation of processes, procedures, and decisions related to assessment design, development and continuous quality monitoring and improvement
• Design and develop valid and reliable assessments that align with learning objectives and competency frameworks.
• Monitors assessment portfolio across the assessment lifecycle
• Manage SOWs for vendor relations
• Collaborates with internal and external partners throughout the design, development, and deployment of assessments.
• Workload management, ability to handle multiple tasks and pay attention to detail
• Individual workload project management
• Staying current with assessment trends, learning science, and industry standards (e.g., AERA, APA, NCME).
• Support the development of digital assets or other collateral as related to assessment.
• As required, provides training, consultation, or guidance to team members and partners on assessment models, assessment design and development practices, and data-driven, evidence-based practices in assessment.
• Performs other related duties as assigned.
• An appetite for curiosity and rapid innovation.

Benefits

• We’re a blended workplace, where team members work remotely or in a hybrid setup depending on their role and location
• We’re mission driven and guided by our culture pillars
• We have a strong commitment to diversity and belonging
• We cultivate a culture of trust, autonomy, and collaboration
• We’re lifelong learners and champion team member growth and advancement
• We’ve got you covered - team member benefits include competitive compensation packages, medical coverage, unlimited PTO, wellness reimbursements, Pluralsight subscription, professional development funds and more.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Number of Employees: 1,001-5,000 employees