Senior Machine Learning Engineer, AI Safety Evaluations

NVIDIA•Santa Clara, CA

1d•$184,000 - $356,500

About The Position

NVIDIA is seeking a Machine Learning Engineer, AI Researcher, or Data Scientist to support AI Safety and Responsibility within Enterprise Risk Management. The role applies data science and statistical skills to convert open research questions into rigorous benchmarks, experiments, and tools for understanding how multimodal large language models behave under pressure. Working with researchers and platform engineers, the position connects evaluation findings to improved training methods and safeguards, producing reproducible evidence for technical reviews and executive decisions.

Requirements

Bachelor’s degree or equivalent experience.
At least 6 years in machine learning engineering or evaluation.
Experience developing benchmarks, metrics, and statistically sound comparisons for machine learning systems, including selecting baselines and interpreting uncertainty.
Ability to translate ambiguous evaluation questions into testable experiments.
Strong Python skills and experience building maintainable shared tools, data pipelines, and experiment-tracking workflows.
Ability to assess dataset quality, identify limitations in metrics and scoring methods, and support conclusions with reproducible evidence.
Ability to clearly explain findings and tradeoffs to researchers, engineering partners, and executives.

Nice To Haves

Experience evaluating LLMs, multimodal models, or agentic systems through adversarial testing or automated red-teaming.
Contributions to public benchmarks or evaluation frameworks.
A publication track record and interest in continuing to publish at top-tier venues.
Experience validating model-based evaluators against human judgments or designing human-in-the-loop evaluation workflows.
Strong model evaluation skills and a commitment to sound statistical practices, including confidence intervals.

Responsibilities

Design and maintain safety benchmarks for large language and multimodal models by translating risk scenarios into test cases, datasets, and scoring rubrics.
Define metrics and protocols that assess safety alongside usefulness, using baselines, uncertainty estimates, and failure analysis to distinguish meaningful improvements from noise.
Build reusable Python evaluation pipelines with versioned datasets, model configurations, and experiment tracking to reproduce results and compare performance across model releases.
Develop automated red-teaming tools to discover failure modes and expand evaluation coverage, incorporating human review to validate automated scoring and provide additional judgment.
Investigate model failures with researchers, document actionable findings, and develop regression tests to assess whether model changes resolve identified issues.
Partner with platform engineers to store and aggregate evaluation results, and communicate findings, limitations, and emerging risks to researchers and decision-makers.