Software Engineer II, Autonomy Evaluation & Metrics Platform

Rivian • Palo Alto, PA

1d

About The Position

Rivian’s Autonomy Eval Infrastructure team builds software that helps engineers understand and improve the machine-learning models behind its driving systems and gives release teams trusted evidence. Within Autonomy’s AI Platform organization, the team connects model development with evaluation workflows, metrics, triage, and visualization across simulation and real-world vehicle data.

The Software Engineer II will build backend services, Python libraries, and data pipelines that turn model outputs into reproducible measurements and useful comparisons. Working with Perception, Planning, and Systems engineers, the role involves learning how models behave, investigating regressions, and using evaluation findings to improve models, data, and release scorecards.

The engineer will own well-scoped components from implementation through deployment and operation, with support from senior engineers on design and complex trade-offs. The newly formed team offers an opportunity to help shape a shared platform while developing software engineering and distributed-systems skills.

Requirements

• At least 2 years of software engineering experience, or equivalent demonstrated experience building and maintaining substantial software systems.
• Strong Python programming skills and a solid foundation in data structures, algorithms, software design, and debugging.
• Practical SQL skills, including the ability to use AI coding tools to support query writing and refinement and to validate query logic, results, and performance.
• Experience building backend services, reusable libraries, or data-processing pipelines with automated tests, version control, and code review.
• Ability to reason about APIs, data models, and component design; explain trade-offs; and identify failure cases such as duplicate, delayed, or missing data.
• Experience with cloud services, containers, and CI/CD, and an understanding of deploying and monitoring software in production.
• Care with data correctness and basic statistical reasoning, including sample sizes, rates, and variability; curiosity about how measurements reflect real driving behavior.
• Clear written communication and a collaborative approach to gathering requirements, asking for feedback, and resolving technical problems.

Nice To Haves

• Hands-on experience with ML models using PyTorch or similar frameworks, such as running inference or analyzing prediction errors; experience in autonomy, robotics, or simulation.
• Experience with distributed data processing or analytical storage, such as Spark, Databricks, Delta Lake, or comparable systems.
• Experience with AWS, Docker, and GitLab CI, or with developer tools, experiment tracking, data lineage, or results visualization.
• Prior autonomy or robotics experience is welcome but not required.

Responsibilities

• Build and test services, APIs, and libraries for registering metric definitions, computing results, and retrieving them consistently across engineering tools.
• Implement scalable metric-processing pipelines with safe retries, incremental computation, and backfills; make missing inputs, failed jobs, and incomplete results visible.
• Capture code, metric, model, dataset, and scenario versions so reported results can be traced to their inputs and reproduced.
• Build tools for comparing model versions and exploring results by scenario, geography, vehicle platform, and operating conditions, using consistent aggregation and denominators.
• Work with model developers to run evaluation experiments, inspect predictions, and investigate failure patterns; help translate findings into targeted evaluation datasets and regression tests.
• Integrate metrics into evaluation workflows, CI regression checks, and release scorecards, and connect results to recordings and triage tools for investigation.
• Improve automated tests, performance, observability, and deployment tooling; debug production issues and document supported components.
• Work with engineers in Palo Alto, London, and Belgrade to turn user needs into maintainable software, participate in design and code reviews, and deliver improvements iteratively.

Benefits

• Medical and prescription drug insurance for full-time employees, their spouse or domestic partner, and children up to age 26; coverage begins on the first day of employment and Rivian covers most premiums.
• Dental insurance for full-time employees, their spouse or domestic partner, and children up to age 26; coverage begins on the first day of employment and Rivian covers most premiums.
• Vision insurance for full-time employees, their spouse or domestic partner, and children up to age 26; coverage begins on the first day of employment and Rivian covers most premiums.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed