Senior Software Engineer, Autonomy Evaluation & Metrics Platform

Rivian • Palo Alto, CA

2d

About The Position

Rivian’s Autonomy Eval Infrastructure team builds software that helps engineers understand and improve the machine-learning models behind its driving systems and provides release teams with reliable evidence. Within the Autonomy AI Platform organization, the team connects model development with evaluation workflows, metrics, triage and visualization across simulation and real-world vehicle data.

The Senior Software Engineer will design, build and operate a substantial part of the autonomy evaluation metrics platform so evaluations are consistent and reproducible. Working with Perception, Planning and Systems engineers, this hands-on role connects model experiments to reliable metrics and turns insights about model behavior, failure modes and generalization into improvements in models and data. The engineer owns projects from design through implementation, rollout and operation; makes practical system-design trade-offs; and mentors teammates. As an early member of a newly formed team, the engineer will also help establish engineering practices and interfaces for broad Autonomy use.

Requirements

• At least 5 years of professional software engineering experience, or equivalent demonstrated experience and impact, including ownership of production backend or data-intensive systems.
• Hands-on experience building software for autonomy or robotics, with direct exposure to evaluation, simulation, perception, planning or validation workflows.
• Strong Python skills, including writing, testing, profiling and debugging maintainable production software.
• Practical SQL skills and ability to validate query logic, results and performance; AI coding tools are expected to support writing and refining queries.
• System-design ability across service APIs, data modeling, distributed processing and storage, including failure recovery, consistency and performance trade-offs.
• Experience taking a project from requirements and design through implementation, rollout and operation, coordinating dependencies and communicating trade-offs clearly.
• Practical understanding of model inference and evaluation, including held-out datasets, generalization, reproducibility and model versioning, and the differences between open-loop replay, closed-loop simulation and real-world measurements.
• Ability to reason about sampling, confidence intervals, noisy metrics and valid comparisons, and to work with domain experts when stronger statistical analysis is needed.
• Experience with cloud infrastructure, containers, CI/CD and production observability.
• Track record of constructive code review and mentoring.

Nice To Haves

• Experience building metric registries, evaluation services, experiment-tracking systems or automated regression gates used by multiple engineering teams.
• Experience with Spark, Databricks, Delta Lake or comparable distributed data systems, and with AWS, Docker or GitLab CI.
• Experience migrating analytical or reporting systems without breaking downstream consumers, or linking metrics to scenario-level debugging and triage.
• Experience with PyTorch or similar ML frameworks, evaluation of end-to-end driving models, or connecting offline model metrics to closed-loop and real-world driving behavior.

Responsibilities

• Own the design and delivery of a metrics-platform area, such as the metric registry, computation services, or comparison and reporting APIs, in partnership with the team’s technical lead.
• Design versioned metric definitions and data models for inputs, aggregation, slicing and results.
• Preserve provenance linking official evaluation runs to code, model, dataset and scenario versions.
• Build distributed computation and storage paths with idempotent execution, incremental processing and controlled backfills.
• Balance throughput, latency, cost and reproducibility in system-design decisions.
• Build comparison tools that expose coverage, sample sizes, uncertainty and changes in evaluation inputs, and help users distinguish model regressions from data, simulator or infrastructure failures.
• Partner with model developers to design evaluation experiments, inspect predictions and analyze failure patterns; use findings to guide model and data improvements and build reusable evaluation harnesses.
• Integrate metrics and regression checks into replay and closed-loop evaluation workflows and CI.
• Partner with domain owners on metric semantics and thresholds, and with Systems on evidence for release decisions.
• Deliver self-service APIs and scorecards backed by consistent definitions and results.
• Migrate existing consumers incrementally while preserving reporting continuity and traceability.
• Own service reliability and performance through testing, observability, incident investigation and operational documentation; measure improvements in turnaround time, correctness and cost.
• Write and review production code, lead design reviews for owned projects, mentor engineers, and collaborate across Palo Alto, London and Belgrade to deliver software users adopt.

Benefits

• Medical and prescription drug insurance; Rivian covers most premiums, with coverage effective on the first day of employment.
• Dental insurance; Rivian covers most premiums, with coverage effective on the first day of employment.
• Vision insurance; Rivian covers most premiums, with coverage effective on the first day of employment.
• Benefits coverage for full-time employees, their spouse or domestic partner, and children up to age 26.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed