Software Engineer, Safeguards Evals

Anthropic•San Francisco, NY

1d•Hybrid

About The Position

Anthropic is seeking a Software Engineer to join their Safeguards Evals team. This role focuses on building and maintaining the evaluation infrastructure for an AI agent that investigates potential misuse of Claude. The engineer will design experiments, build datasets representing real abuse, and implement methods into pipelines that govern system changes. The work directly impacts the trust in automated abuse detection and guides improvements. The position is at the intersection of applied ML research and engineering.

Requirements

Proficiency in Python and comfort working across the stack.
Experience building and maintaining data pipelines.
Experience working with LLMs and a working understanding of their capabilities and failure modes, especially agentic systems with tool use and multi-step reasoning.
Strong data analysis skills.
Ability to move fluidly between research prototyping and production-quality code.
Ability to translate ambiguous problems into concrete, testable experiments.

Nice To Haves

6+ years of industry software engineering experience.
Expertise in building or contributing to agent evaluation frameworks, benchmarks, or automated grading systems.
Extensive experience in trust and safety, content moderation, or abuse detection systems.
Experience in red teaming, adversarial testing, or jailbreak research on AI systems.
Experience with synthetic data generation or data augmentation.
Experience with distributed systems or large-scale data processing.
Experience with prompt engineering or building LLM-powered applications.

Responsibilities

Build and own the evaluation harness for an agentic investigation system, defining metrics, test cases, and grading approaches.
Construct high-quality eval datasets representing real-world misuse across harm areas, drawing from real traffic patterns and synthetic generation.
Measure agent performance end-to-end (detection precision/recall, investigation quality, robustness) and drive improvements.
Analyze coverage to identify measurement gaps and evolve evals to remain unsaturated and high-signal.
Productionize successful research into regression and release pipelines that run on every agent change, prompt update, and underlying model upgrade.
Build tooling that enables policy experts to author, run, and iterate on evaluations without engineering support.
Construct RL environments to improve Claude’s safety investigation capabilities.