Member of Technical Staff, Research Scientist - Safety Environments

Plato • San Francisco, CA

1d • $275,000 - $450,000

About The Position

Plato is hiring an individual contributor researcher to establish a research direction focused on AI safety environments and benchmarks. The initial focus is evaluating AI research and self-improvement capabilities, including experiment design, model training, and research replication. The role also investigates reward hacking, sabotage, deception, and other misaligned behavior through research design, empirical evaluation, and failure analysis. The researcher will own substantial parts of the research agenda and work with engineers to convert experimental designs into runnable, reproducible environments. Some environments may also be used in training to reduce the behaviors they reveal. The work includes evaluating frontier models and communicating findings through benchmark releases, reports, and recommendations for lab partners.

Requirements

• Hands-on work in frontier-model evaluation, benchmarks, or AI safety within the past one to two years.
• Meaningful contributions to papers, released benchmarks, public reports, or comparable empirical work; quality and contribution matter more than credentials or years of experience.
• Practical familiarity with current frontier models and a specific, evidence-based view of their capabilities, limitations, and likely development.
• Ability to explain what an evaluation measures, how it could be gamed, and which conclusions its results do and do not support.
• Ability to turn an ambiguous question into a concrete experimental design and collaborate closely with engineers to implement it.
• Interest in measuring safety-relevant capabilities and behavior, including tradeoffs when evaluation environments are also used for training.
• Based in the San Francisco Bay Area or willing to relocate.

Nice To Haves

• Designed and released a benchmark, particularly for AI R&D, long-horizon agents, dangerous capabilities, or AI control; reference points include RE-Bench, MLE-bench, and PaperBench.
• Experience in an independent evaluation or safety organization, a frontier lab safety team, or pre-deployment testing with labs or government evaluation bodies.
• Direct research on models conducting ML experiments, training or improving other models, replicating papers, or completing extended research tasks.
• Software engineering background or programming skills to implement environments, evaluation harnesses, and analysis tools.
• Experience publishing rigorous empirical research, including work at major machine-learning venues.

Responsibilities

• Design benchmarks that measure AI research automation and self-improvement, defining tasks, baselines, budgets, and success criteria.
• Build environments that expose reward hacking, sabotage, deception, and failures to follow intended constraints.
• Assess evaluations for contamination, grader weaknesses, task ambiguity, and shortcuts that could make results misleading.
• Run experiments with current models, inspect trajectories and failures, and measure capability changes across models and over time.
• Collaborate with engineers to turn research designs into reproducible environments with clear scoring, useful instrumentation, and documented limitations.
• Translate results into benchmark releases, research reports, and concrete recommendations for lab partners.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed