Staff+ Software Engineer, Safeguards Evals

Anthropic•New York City, NY

14h•Hybrid

About The Position

Anthropic is hiring a Staff+ Software Engineer to build methods and infrastructure for trustworthy safety evaluations. The role sits at the intersection of applied machine learning research and engineering, improving evaluations of model behavior and the agentic systems that govern it. The work includes designing experiments, building datasets representative of real-world misuse, and integrating successful methods into pipelines used for model training, agent changes, and launch decisions. The role also develops evaluations and reinforcement-learning environments for an agentic investigation system that analyzes real-world traffic to identify potential misuse and emerging threats.

Requirements

Experience building and maintaining data pipelines.
Experience working with LLMs and a working understanding of their capabilities and failure modes, especially agentic systems with tool use and multi-step reasoning.
Strong data analysis skills and the ability to draw reliable insights from large datasets.
Ability to move between research prototyping and production-quality code.
Ability to translate ambiguous problems into concrete, testable experiments.
Care deeply about AI safety and want work to have real impact.
At least a bachelor's degree or an equivalent combination of education, training, and/or experience.
A field of study relevant to the role, as demonstrated through coursework, training, or professional experience.

Nice To Haves

8 or more years of industry software engineering or machine learning engineering experience.
Expertise building or contributing to LLM or agent evaluation frameworks, benchmarks, or automated grading systems.
Extensive experience in trust and safety, content moderation, or abuse detection systems.
Experience in red teaming, adversarial testing, or jailbreak research on AI systems.
Experience with synthetic data generation or data augmentation.
Experience with distributed systems or large-scale data processing.
Experience with prompt engineering or building LLM-powered applications.

Responsibilities

Design and run experiments to improve evaluation quality, including methods for generating representative test data, simulating realistic user behavior, and validating grading accuracy.
Research how multi-turn conversations, tools, long context, and user diversity affect model safety behavior.
Analyze evaluation coverage, identify measurement gaps, and evolve evaluations to remain unsaturated and high-signal as model and agent capabilities advance.
Build and own the evaluation harness for an agentic investigation system, including metrics, test cases, and grading approaches.
Measure agent performance end-to-end, including detection precision and recall, investigation quality, and robustness, and drive improvement in the hardest harm areas.
Construct reinforcement-learning environments to improve Claude's safety investigation capabilities.
Build high-quality evaluation datasets representing real-world misuse, including cyber attacks, biological weapons, and influence operations, using real traffic patterns and synthetic generation.
Collaborate with Policy and Enforcement to translate observed harm patterns into measurable evaluations.
Ship research into evaluation, regression, and release pipelines used during model training, agent changes, prompt updates, model upgrades, and after launch.
Build tools that let policy experts author, run, and iterate on evaluations without engineering support.
Share findings with research and training teams to drive upstream model improvements.