Evaluation and ML Systems Engineer, AI Safety and Security Engineering

NVIDIA•Santa Clara, TN

8d•$152,000 - $241,500

About The Position

NVIDIA is seeking an Evaluation/ML-Systems Engineer to join their AI Safety & Security Engineering team. This role is crucial for measuring the program's effectiveness, ensuring that real capability is distinguished from anecdote and that progress is quantifiable. The engineer will design metrics, establish baselines, and create reproducible protocols for fair comparisons. They will build the infrastructure to track results, link them to their origins (code and runs), and facilitate verification of surprising findings. The goal is to automate measurement processes, allowing researchers to focus on analysis. This role involves partnering with security researchers and platform engineers to design trustworthy and repeatable experiments, supporting internal reviews and fostering an evidence-based approach within the team.

Requirements

Bachelor's degree (or equivalent experience) with 5+ years in ML engineering or evaluation.
Evaluation experience: Designing benchmarks, metrics, and statistically sound comparisons for ML systems.
Measurement rigor: A careful, skeptical approach to metrics, baselines, and claims.
Engineering skills: Solid Python engineering for shared infrastructure, including experiment tracking and data pipelines.

Nice To Haves

Security evaluation: Exposure to evaluating security tooling or pipelines.
Agentic systems: Experience measuring agent or LLM behavior.
Community work: Contributions to public benchmarks or evaluation frameworks.

Responsibilities

Build the benchmarking and reproducibility systems for evaluation infrastructure.
Define the metrics and protocols against which measurements are made.
Map every result to the code and runs that produced it for traceability.
Ensure findings are reviewable and conclusions are traceable to maintain evidence discipline.