Evaluation Engineer

Maven Robotics • Santa Clara, CA

11h

About The Position

As a Robotics Evaluation Engineer at Maven Robotics, you will build the systems used to measure, diagnose, and improve robot policy performance across simulation and real robots. You will own the core evaluation infrastructure that enables the AI team to compare models rigorously, understand failure modes, measure generalization and reliability, and make better decisions about data, training, and deployment.

You will work closely with robot learning researchers and engineers across simulation, perception, controls, data, and infrastructure. Together, you will define task success, build reproducible evaluation frameworks, automate testing, and develop tools to understand why robot policies succeed or fail.

Requirements

• Significant experience building robotics evaluation, simulation, testing, or benchmarking systems.
• Strong programming ability in Python, C++, or similar robotics or systems languages.
• Experience with robotics simulation platforms such as Isaac Sim, MuJoCo, Gazebo, or similar systems.
• Experience evaluating learned robotic policies in simulation and on physical robots.
• Experience designing reproducible experiments, metrics, benchmarks, and failure-analysis workflows.
• Ability to build reliable software systems for automated testing, data collection, visualization, and reporting.
• Strong understanding of robotics systems and ability to debug issues across software, models, sensors, controls, and hardware.
• Track record of independently owning complex technical projects from initial design through deployment and ongoing operation.

Nice To Haves

• Familiarity with sim-to-real transfer, domain randomization, sensor modeling, and system identification.
• Experience importing and configuring simulation assets, including robot models, objects, joints, collisions, inertial properties, materials, and sensors.
• Experience with hardware-in-the-loop testing, continuous integration for robotics, or automated robot test cells.
• Familiarity with multimodal robot policies, vision-language-action models, or other learned manipulation architectures.
• Strong background in statistical analysis, confidence estimation, and experimental design.

Responsibilities

• Own the architecture, implementation, and evolution of Maven’s robot learning evaluation framework across simulation and real robots.
• Define task specifications, success criteria, failure taxonomies, evaluation metrics, and benchmark protocols for manipulation policies.
• Build automated systems for comparing models, detecting regressions, running large-scale evaluations, and generating performance reports.
• Develop simulation environments, task scenarios, and procedural variations for policy evaluation.
• Build reproducible real-robot evaluation workflows that minimize manual supervision and produce statistically meaningful results.
• Measure and improve the relationship between simulated and real-world policy performance.
• Build tools for replaying, visualizing, and diagnosing policy failures across simulation and physical robots.
• Work with robot learning researchers to identify whether failures originate from the model, data, perception, controls, calibration, simulation, hardware, or long-horizon execution.

What This Job Offers

• Job Type: Full-time
• Education Level: No Education Listed