Staff Engineer, Autonomy Evaluation (VLA)

Cariad, Inc.•Mountain View, CA

20h•Hybrid

About The Position

We are CARIAD, an automotive software development team with the Volkswagen Group. Our mission is to make the automotive experience safer, more sustainable, more comfortable, more digital, and more fun. To achieve that we are building the leading tech stack for the automotive industry and creating a unified software platform for over 10 million new vehicles per year. We’re looking for talented, digital minds like you to help us create code that moves the world. Together with you, we’ll build outstanding digital experiences and products for all Volkswagen Group brands that will transform mobility. Join us as we shape the future of the car and everyone around it. Role Summary: The Staff Engineer, Autonomy Evaluation (VLA) leads the development of evaluation methods, metrics, and tooling for VLA / VLAM-based automated driving at CARIAD. This role turns model behavior into clear performance evidence across route-level, scenario-level, and regression testing. You will build the evaluation loop that connects data, simulation, model iteration, and in-vehicle performance, helping the team improve driving quality quickly and make confident technical decisions. You will work closely with ML, systems, and integration engineers to define how progress is measured for both demo performance and future productization.

Requirements

Systems-level thinking and ability to translate ambiguous driving behavior into measurable outcomes
Strong analytical skills in benchmarking, regression analysis, and experiment design
Ability to connect quantitative findings to practical engineering decisions and release readiness
Strong cross-functional communication skills; comfortable working across teams and time zones
High ownership mindset with rigor, curiosity, and a bias toward action
Deep understanding of autonomy evaluation for ADAS / AD, robotics, or embodied AI systems
Strong experience with performance metrics, benchmarking, regression analysis, and scenario-based validation
Strong software engineering skills in Python; experience building robust evaluation or data pipelines
Experience working with simulation, large-scale driving data, or model-in-the-loop evaluation
8+ years of experience in autonomy, robotics, machine learning, or evaluation for complex technical systems
3+ years of experience building metrics, validation frameworks, or evaluation pipelines for ML-based systems

Nice To Haves

Experience with foundation models, VLMs / VLAMs, or end-to-end driving models
Familiarity with learned evaluators, LLM/VLM-assisted evaluation, or agentic workflows for analysis and triage
Experience in autonomous driving, robotics, or other safety-critical systems

Responsibilities

Define and evolve evaluation strategies for VLA / VLAM-based driving performance
Design metrics and success criteria spanning safety, comfort, robustness, and generalization
Create benchmarks for route-level, scenario-level, and long-tail/edge-case behavior
Develop scalable evaluation pipelines across offline logs, simulation, and in-vehicle test results
Build tooling for dataset slicing, scenario curation, and automated report generation
Enable reproducible evaluation with clear versioning of data, models, and metrics
Detect and diagnose regressions across model iterations and software releases
Establish a structured forensic / root-cause analysis workflow for driving failures and edge cases
Translate findings into actionable improvements for model training, data collection, and system integration
Partner with ML and integration engineers to assess model quality and prioritize fixes
Support release and demo readiness decisions with clear, quantitative evidence
Communicate results and trade-offs to technical and non-technical stakeholders
Define best practices for evaluation methodology, metrics hygiene, and reporting
Mentor engineers and raise the bar for rigor and speed of iteration in the evaluation loop
Contribute to long-term evaluation architecture for productization