Principal Engineer, Behavior Understanding and Evaluation

Motional•Boston, MA

14d•Hybrid

About The Position

The Behavior Understanding and Evaluation team at Motional is dedicated to comprehending and assessing the intricate behavior of the Motional Vehicle. To prepare for unprecedented scale, it's paramount that we leverage automated algorithms to thoroughly understand and evaluate our vehicles' performance, both in real-world deployment and within simulated environments. We aim to employ cutting-edge machine learning techniques in our evaluation processes, ensuring we can scale effectively to meet Motional's ambitious goals. Our collaborative approach extends across various teams, providing critical insights and actionable metrics to those developing the onboard software and enhancing the rider experience. We are responsible for building and operating robust, large-scale machine learning and data systems, alongside sophisticated simulation workflows and insightful diagnostic tools. By combining expert human judgment with advanced machine learning models, we generate essential training and evaluation data for the Motional driver. We are seeking passionate researchers and software engineers to join us in building production-grade machine learning evaluation systems for autonomous vehicles - people who are driven to transform complex data into measurable progress toward a safer, smarter transportation future.

Requirements

Proven experience in establishing and executing evaluation strategies.
A strong foundation in scientific and statistical methodologies.
Expertise in Machine Learning and Deep Learning.
At least 3 years of experience leading and scaling large-scale data and evaluation systems.
Proficiency in crafting data-driven reports and technical strategy documents for executive decision-making.
Technical leadership experience, including hands-on involvement in the development of large-scale AI or VLM/LLM systems.

Nice To Haves

Experience in mentoring and leadership development.
A background in shaping product roadmaps.
Familiarity with Reinforcement Learning (RL) with Human Feedback.
Specialized expertise in creating expansive datasets.
Experience with large-scale global consumer products.

Responsibilities

Define the strategic direction for the development and application of machine learning-driven evaluation methodologies for autonomous driving.
Optimize large-scale evaluation infrastructure to ensure scalability.
Partner cross-functionally with Behaviors, Actions, Simulation, System Engineering, Research, and other Onboard/Offboard Software Teams on evaluation strategies.
Upgrade evaluation pipelines by enhancing existing Heuristic and Event based processes with learned methods
Uphold scientific and statistical rigor in the application of embodied AI.
Pioneer the definition and implementation of key evaluation metrics and reward functions, , driving autonomy performance across safety, comfort, and assertiveness.
Align stakeholders around clear evaluation objectives and priorities.