Lead Specialist, AI Scientist
About The Position
Pearson is seeking a Lead Specialist, Applied AI Systems Engineer to design, build, and continuously improve intelligent systems supporting the development of K–12 assessment and learning. The role sits within the Measurement & Learning team and works across content development, learning science, technology, AI science, and psychometrics. It partners with software engineers and the AI Data & Knowledge Architect to translate assessment and measurement expertise into testable AI capabilities for Pearson’s assessment authoring system, which supports formative and summative assessment. This hands-on role focuses on building and extending complete AI systems on a shared knowledge foundation, rather than concentrating only on prompts or chatbots. The work may involve predictive machine learning, statistical methods, retrieval, graph-based approaches, generative and agentic AI, and deterministic software, selected according to reliability and evidence. The role emphasizes end-to-end system design, knowledge-grounded and agentic AI, evaluation-driven development, model adaptation, responsible AI, and cross-functional collaboration. It requires building systems whose quality can be measured, explained, audited, and continuously improved, while supporting rather than supplanting human expertise. The position will help develop the intelligent core of an assessment system for formative and summative assessment.
Requirements
- Advanced degree in Computer Science, Artificial Intelligence, Machine Learning, Statistics, or a related field, or equivalent experience.
- Experience designing and building complete AI or machine-learning systems end to end, not only isolated models or prompts.
- Strong breadth across predictive machine learning, statistical learning, and LLM-based, agentic, and retrieval-grounded systems, with judgment about when each is appropriate.
- Demonstrated experience evaluating probabilistic systems against ground truth or expert judgment, including evaluation datasets, error analysis, and regression testing.
- Experience with knowledge graphs and knowledge-grounded AI sufficient to contribute to architecture decisions.
- Strong Python proficiency and experience with TypeScript or the ability to become productive quickly in a TypeScript codebase.
- Track record of innovation beyond established practice, sound technical judgment, and high autonomy.
- Ability to collaborate closely with engineers and domain experts, translating expertise into requirements and evaluation criteria.
Nice To Haves
- Experience building knowledge and inference layers of education, learning, or assessment systems, such as skill or competency graphs, learner or domain models, standards-alignment engines, adaptive or recommendation logic, or inference behind learner insight reporting.
- Experience building AI or machine-learning systems in evidence-sensitive domains such as scientific, healthcare, financial, legal, or assessment applications.
- Hands-on experience with parameter-efficient fine-tuning or other model-adaptation methods.
- Experience designing, filtering, and validating synthetic data for evaluation or model improvement.
- Experience building expert-annotated datasets, adjudication protocols, or evaluation platforms.
- Experience with graph-grounded agents, hybrid retrieval, or other knowledge-intensive AI applications.
- Familiarity with ontologies, semantic models, or hybrid neural-symbolic architectures.
- Experience with calibration, uncertainty estimation, drift detection, or model monitoring.
- Experience with TypeScript, AWS, PostgreSQL, and graph data stores; the environment also uses Go, Python, and ML lifecycle tools such as MLflow.
- Familiarity with educational measurement, psychometrics, or learning science.
Responsibilities
- Design modular end-to-end AI systems spanning knowledge and data inputs, retrieval, models, agents, tools, deterministic services, human review, evaluation, and integration.
- Break complex problems into appropriate combinations of predictive machine learning, statistical methods, generative models, retrieval, graph reasoning, deterministic software, and human decision points.
- Build working systems and reference implementations, and partner with software engineers to integrate them into the assessment authoring system.
- Design architectures that can evolve as models, data, domain knowledge, and evaluation evidence change.
- Design and implement agentic and multi-step workflows using models, tools, retrieval, state, structured outputs, and human decision points.
- Build knowledge-grounded systems using graph, semantic, vector, or hybrid retrieval as appropriate.
- Co-design knowledge and graph architecture with the AI Data & Knowledge Architect and contribute substantively to the knowledge layer.
- Develop AI capabilities for content creation and revision, verification, framework alignment, guidance, and analytic insights.
- Build and maintain evaluation benches, including expert-adjudicated datasets, challenge sets, regression suites, and protected holdout data.
- Work with psychometricians and content experts to translate domain expectations into measurable evaluation criteria.
- Analyze failure modes, critical errors, and performance across meaningful data and task slices, not only aggregate metrics.
- Monitor deployed systems for drift and emerging failures, and use findings to drive improvement.
- Develop predictive models, classifiers, and ranking methods when they outperform generative AI.
- Evaluate tradeoffs among prompting, retrieval, structured knowledge, model selection, and model adaptation without locking the architecture to a single provider.
- Apply parameter-efficient fine-tuning and other adaptation methods when evaluation evidence shows they add value.
- Turn expert judgments and adjudicated examples into versioned evaluation and training assets while keeping training data separate from protected evaluation data.
- Design synthetic-data approaches with rigorous quality controls and avoid treating generated data as ground truth by default.
- Prototype quickly, test assumptions empirically, and pivot when evidence warrants.
- Build accurate, transparent, auditable AI designed to support rather than supplant human expertise.
- Uphold responsible AI and data practices, including privacy, security, fairness, and appropriate use of student and content data.
- Partner with Technology teams and software engineers to integrate AI capabilities into the assessment authoring system.
- Collaborate with AI Science colleagues and the AI Data & Knowledge Architect on shared architecture and methods.
- Work with psychometricians and learning scientists to translate measurement and learning requirements into evaluation criteria.
- Partner with content development teams to understand workflows, quality requirements, edge cases, and human-review needs.
- Communicate findings, tradeoffs, and limitations clearly to technical and non-technical audiences.
Benefits
- Eligible to participate in an annual incentive program.
- Benefits information is available at Pearson Benefits Global.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
