Member of Technical Staff, Research Scientist - Safety Environments

Plato • San Francisco, CA

16d

About The Position

Plato is hiring an individual contributor researcher to establish a research direction focused on AI safety environments and benchmarks. The role begins with evaluations of how well AI models can improve themselves and other models, including AI research and development capabilities such as experiment design, model training, and research replication. The researcher will also investigate reward hacking, sabotage, deception, and other misaligned behavior.

The work spans research design, empirical evaluation, and failure analysis, with substantial ownership of the research agenda. The researcher will define benchmark tasks, baselines, budgets, and success criteria; assess whether evaluations produce meaningful evidence; and work closely with engineers to make research designs into reproducible, runnable environments. Some environments may also be used in training to reduce the behaviors they reveal. Findings will be shared through benchmark releases, research reports, and recommendations for lab partners.

Requirements

• Hands-on work in frontier-model evaluation, benchmarks, or AI safety within the past one to two years.
• Meaningful contributions to papers, released benchmarks, public reports, or comparable empirical work; quality of work and contribution matter more than credentials or years of experience.
• Practical familiarity with current frontier models and a specific, evidence-based view of their capabilities, limitations, and likely development.
• Ability to explain what an evaluation measures, how it could be gamed, and which conclusions its results do and do not support.
• Ability to turn an ambiguous question into a concrete experimental design and collaborate closely with engineers to implement it.
• Interest in measuring safety-relevant capabilities and behavior, including tradeoffs when evaluation environments are also used for training.
• Based in the San Francisco Bay Area or willing to relocate.

Nice To Haves

• Designed and released a benchmark, particularly for AI R&D, long-horizon agents, dangerous capabilities, or AI control; reference points include RE-Bench, MLE-bench, and PaperBench.
• Experience in an independent evaluation or safety organization, a frontier lab safety team, or pre-deployment testing with labs or government evaluation bodies.
• Direct research on models conducting ML experiments, training or improving other models, replicating papers, or completing extended research tasks.
• Software engineering background or programming skills to implement environments, evaluation harnesses, and analysis tools.
• Experience publishing rigorous empirical research, including work at major machine-learning venues.

Responsibilities

• Design safety benchmarks that measure AI research automation and self-improvement, including tasks, baselines, budgets, and success criteria.
• Build safety environments that expose reward hacking, sabotage, deception, and failures to follow intended constraints.
• Investigate contamination, grader weaknesses, task ambiguity, and shortcuts that could make evaluation results misleading.
• Run experiments with current models, inspect trajectories and failures, and measure capability changes across models and over time.
• Work with engineers to implement research designs as reproducible environments with clear scoring, useful instrumentation, and documented limitations.
• Translate findings into benchmark releases, research reports, and concrete recommendations for lab partners.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed