About The Position

Apple Services Engineering is building the scientific foundation for evaluating AI systems across Apple. The Measurement Scientist will establish scientifically valid, trustworthy measurement standards and apply state-of-the-art measurement methods, validity frameworks, experimental design, and statistical rigor to determine what AI evaluations measure, whether changes help, why systems fail, and whether they are ready to ship. The role bridges measurement science and AI evaluation. The scientist will develop methods for validating AI evaluation tools, artifacts, and workflows so developers receive reliable signals for sound engineering decisions. They will collaborate with measurement scientists, ML researchers, platform engineers, and evaluation practitioners to translate measurement science into practical tools that scale across Apple. Work will be published at leading measurement and ML venues and incorporated into the evaluation SDK used across Apple. The role is open to candidates across a range of experience levels.

Requirements

  • PhD in Computer Science or Machine Learning with a measurement or evaluation focus, Psychometrics, Educational Measurement, Quantitative Psychology, Statistics, Biostatistics, Econometrics, or equivalent research/work experience.
  • Deep expertise in at least one of: modeling test or benchmark data (IRT, latent variable, or other response models) and construct validation; benchmark design and validity for AI systems; or experimental design and causal inference, including surrogate outcomes and observational causal methods.
  • Strong statistical foundation, including experimental design, power analysis, sampling theory, and uncertainty quantification.
  • Track record of designing and validating measurement instruments or experiments, demonstrated through publications or applied work.
  • Proficiency in Python (preferred) or R for statistical analysis, psychometric modeling, and method implementation.
  • Strong working knowledge of generative AI technology.
  • Excellent communication skills, including the ability to explain complex measurement concepts to engineers, ML researchers, and non-technical stakeholders.

Nice To Haves

  • Research or experience applying measurement science to AI/ML evaluation, benchmark validity, automated scoring systems, or computational assessment.
  • Knowledge of modern ML evaluation challenges, including LLM-as-judge, automated metrics, benchmark design, and agentic systems.
  • Publications at measurement venues or top ML conferences such as NeurIPS, ICML, or ICLR.
  • Expertise in computational social or behavioral science using generative AI.
  • Experience collaborating with engineers to turn research methods into production tools and scalable infrastructure.
  • Experience with online experimentation or validating offline metrics against real-world outcomes using surrogate or proxy metrics.
  • Experience estimating rare events or making decisions when one kind of error costs far more than another.

Responsibilities

  • Apply measurement methods, validity frameworks, experimental design, and statistical rigor to AI evaluation.
  • Establish standards that ensure AI evaluators validly measure constructs such as helpfulness and safety.
  • Develop methods for validating AI tools, evaluation artifacts, and workflows used by developers.
  • Collaborate with measurement scientists and ML researchers on AI evaluation problems.
  • Translate measurement science into practical, scalable tools in collaboration with platform engineers and evaluation practitioners.
  • Publish research and productionize methods in Apple's evaluation SDK.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service