Magic’s mission is to build safe AGI that accelerates humanity’s progress on important problems. Its approach combines frontier-scale pretraining, domain-specific reinforcement learning, ultra-long context, and inference-time compute. The Evals team measures the performance and capabilities of internal and external models, supporting pretraining, RL, data, inference, and product teams. As a Member of Technical Staff on Evals, you will develop benchmarks and evaluation methodologies, run data ablations, assess dataset quality, and investigate whether measurements accurately reflect model performance. The team emphasizes correctness, reproducibility, and measurement quality, including finding subtle bugs and flawed assumptions in benchmarks, papers, and open-source evaluation frameworks.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Job Type
Full-time
Career Level
Mid Level
Education Level
No Education Listed