Member of Technical Staff, Evals

Magic•San Francisco, CA
•$200,000 - $550,000

About The Position

Magic’s mission is to build safe AGI that accelerates humanity’s progress on important problems. Its approach combines frontier-scale pretraining, domain-specific reinforcement learning, ultra-long context, and inference-time compute. The Evals team measures the performance and capabilities of internal and external models, supporting pretraining, RL, data, inference, and product teams. As a Member of Technical Staff on Evals, you will develop benchmarks and evaluation methodologies, run data ablations, assess dataset quality, and investigate whether measurements accurately reflect model performance. The team emphasizes correctness, reproducibility, and measurement quality, including finding subtle bugs and flawed assumptions in benchmarks, papers, and open-source evaluation frameworks.

Requirements

  • Experience developing model evaluations, benchmarks, or machine learning experiments.
  • Strong programming skills and experience working with machine learning systems.
  • Track record of owning technical projects end-to-end.
  • Skepticism toward results that cannot be reproduced, validated, or explained.
  • Experience analyzing model performance and interpreting experimental results.
  • Comfort navigating ambiguity and determining whether a measurement captures the behavior it claims to measure.
  • Commitment to raising the standard of rigor in AI evaluations.

Responsibilities

  • Design and validate evaluations and benchmarks across model training and deployment.
  • Investigate model behavior and identify gaps in existing evaluations.
  • Measure dataset quality and identify opportunities to improve training data.
  • Improve evaluation correctness, reproducibility, and reliability.
  • Audit and improve public benchmarks, research papers, and open-source implementations.
  • Define metrics that accurately reflect model quality.
  • Build tools that make evaluation results accessible and useful across Magic.

Benefits

  • Annual salary range of $200K–$550K, depending on experience.
  • Equity as a significant part of total compensation, in addition to salary.
  • 401(k) plan with 6% salary matching.
  • Health, dental, and vision insurance for the employee and dependents.
  • Unlimited paid time off.
  • Visa sponsorship and relocation support for candidates moving to San Francisco.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service