Lead Specialist, Psychometrician

Pearson•United States,

2d•$130,000 - $170,000

About The Position

We are seeking a Lead Specialist, Psychometrician to support the design, validation, and continuous improvement of intelligent measurement systems and AI-assisted assessment and learning workflows. This role sits within the Measurement & Learning team and operates at the intersection of Measurement, Content Development, Learning Science, Technology, and AI Science, with responsibility for helping ensure AI-assisted systems consistently produce high-quality, valid, scalable, and instructionally meaningful outputs across assessment and learning use cases. The role primarily supports AI-assisted assessment item development workflows, while also contributing to adjacent applications including AI-generated practice items, learning materials aligned to standards, AI-powered learning and feedback experiences, instructional guidance systems, and emerging approaches to evidencing student skills and competencies. This role also contributes to the advancement of next-generation assessment approaches, including innovative item types, AI-assisted measurement opportunities, and new ways of assessing reasoning, applied learning, and student competencies.

Requirements

Advanced degree in Educational Measurement, Psychometrics, Learning Sciences, Educational Technology, Artificial Intelligence, Educational Data Mining, Learning Analytics, Cognitive Science, Computer Science, Statistics, or a related field
Experience working with assessment item development, learning systems, or educational content workflows
Experience collaborating across cross-functional environments including content, measurement, technology, product, or AI-related teams
Strong understanding of assessment quality principles, including validity, alignment, fairness, accessibility, and instructional appropriateness
Ability to think systematically about AI-assisted workflows across content generation, scoring, feedback, instructional guidance, and learner-facing experiences
Ability to evaluate where AI-assisted approaches can effectively support assessment and learning workflows and where human expertise remains essential
Strong analytical and problem-solving skills with attention to quality, defensibility, and continuous improvement
Ability to design or support rigorous validation and monitoring approaches for AI-assisted systems and workflows
Comfort exploring, testing, and prototyping AI-assisted approaches using emerging tools and technologies
Strong written and verbal communication skills, including the ability to translate complex AI and measurement concepts for diverse audiences

Nice To Haves

Experience supporting AI-assisted assessment, scoring, feedback, tutoring, instructional guidance, or content-generation systems
Experience designing or evaluating validation studies for AI-assisted educational systems or workflows
Familiarity with psychometric principles and educational measurement concepts
Experience with innovative item types, technology-enhanced items (TEIs), oral or constructed-response assessment, or competency-based assessment approaches
Experience contributing to technical documentation, dissemination materials, presentations, publications, or thought leadership initiatives
Familiarity with recommender systems, learner models, domain models, or AI-guided instructional experiences
Experience identifying workflow risks, edge cases, drift concerns, or unintended consequences within AI-assisted systems
Experience leveraging AI-assisted tools to accelerate workflow exploration, prototyping, innovation, or analysis
Curiosity about emerging approaches to assessing skills, competencies, reasoning, and applied learning through AI-assisted systems
Experience operating in fast-moving innovation environments where systems and workflows continuously evolve

Responsibilities

Lead and support the design and continuous improvement of AI-assisted assessment item development workflows embedded within broader assessment and learning systems
Translate SME and measurement intent into structured, reusable AI-assisted workflows and agentic frameworks that support scalable, consistent, and auditable content generation and enhancement
Use AI-assisted tools and rapid prototyping approaches to explore, test, and iterate on emerging assessment, content, learning, and instructional workflow innovations
Support assessment and learning content across Mathematics, English Language Arts, Science, Social Studies, K–12 grade levels, TEIs, oral and constructed-response interactions, translation and multilingual content, and alignment to academic content standards
Support fact checking, grammar, style refinement, distractor rationale strengthening, and key validation
Ensure AI-assisted outputs meet expectations for validity, accuracy, fairness, accessibility, and instructional appropriateness
Lead and support validation approaches for AI-assisted learning and assessment applications beyond core assessment item development workflows including Generation and evaluation of academically similar items and learning opportunities for student practice
AI-assisted learning materials aligned to standards, learning goals, and instructional intent
Innovative item types and new ways of assessing student skills, competencies, reasoning, communication, and applied learning
Support validation and quality approaches for content and interactions used within AI-assisted online learning guides, instructional supports, and learner-facing experiences
Help ensure AI-assisted assessment, feedback, and learning supports work together in educationally meaningful and instructionally aligned ways
Contribute to the advancement of emerging intelligent measurement and learning system capabilities across assessment and instructional contexts
Design and oversee validation studies evaluating the quality, appropriateness, effectiveness, interpretability, and educational defensibility of AI-assisted systems, outputs, and workflows
Define evaluation criteria in collaboration with psychometricians, learning scientists, and content leaders
Establish processes for ongoing monitoring and continuous improvement
Develop frameworks for ongoing monitoring, drift detection, and continuous improvement of AI-assisted systems over time
Support research and validation frameworks grounded in human expertise, empirical evidence, transparency, and continuous improvement
Support technical documentation and evidence generation related to the validity, quality, effectiveness, interpretability, and performance of AI-assisted systems and outputs
Contribute to internal and external thought leadership related to intelligent measurement systems and AI-assisted learning technologies
Support dissemination of research findings, validation approaches, and innovation frameworks through technical reports, presentations, and publications
Help translate complex AI, learning, and measurement concepts into clear guidance for diverse stakeholder groups
Provide guidance to teams on responsible and effective use of AI-assisted systems within assessment and learning workflows
Partner with Technology teams implementing AI infrastructure and intelligent systems
Partner with AI Science teams building models and systems for scoring, reporting, feedback, guidance, and learning supports
Partner with content teams authoring assessment and learning content
Collaborate with research and measurement teams supporting learner models, recommender systems, and AI-guided instructional experiences
Ensure alignment between AI-assisted content generation, downstream scoring systems, instructional guidance systems, and learner-facing experiences
Support coherence across assessment, learning, scoring, feedback, and instructional systems