Machine Learning Engineer - Vision Foundation Models

Zoox•San Diego, CA

9d•$254,000 - $305,000

About The Position

The Perception team at Zoox develops the robotaxi’s ability to understand the world, including the vehicle cabin. This role designs, trains, and evaluates vision-based in-cabin perception models that understand passengers, objects, and behaviors. These models support rider safety and the operations needed to keep rides running smoothly. The engineer independently owns a model or feature area end to end, from data curation and training through evaluation and deployment, working with partners across the company. The work has a direct impact on rider safety and experience as Zoox scales its robotaxi service.

Requirements

MS or PhD in Computer Science, Machine Learning, Computer Vision, or a related field, with 6+ years of industry experience; PhD holders require 3+ years.
Deep expertise in computer vision and deep learning for image and video understanding, including object detection, pose and action recognition, and fine-grained classification.
Hands-on experience with modern architectures, including convolutional and vision-transformer detectors and self-supervised or foundation-model backbones such as DINOv3.
Strong proficiency in Python and PyTorch for model design, training, and evaluation.
Experience with vision-language models or foundation-model backbones, and with auto-labeling or synthetic data for long-tail and rare classes.
Track record of owning the full ML lifecycle on real-world data, including dataset curation, labeling guidelines, training, metric design (precision, recall, and false-positive rate at an operating point), and error analysis.

Nice To Haves

Experience with in-cabin or occupant monitoring systems, ideally including multi-camera or fisheye imagery and low-light conditions.
Experience deploying deep learning models to real-time, resource-constrained systems such as robotics, automotive, or edge devices, using TensorRT, ONNX, or similar.
Proficiency in C++ for model integration.

Responsibilities

Design, train, and evaluate vision models for understanding passengers, objects, and behaviors inside the vehicle, including detection, pose, and action recognition.
Own a model or feature area end to end by curating data, defining labeling guidelines, training and evaluating models, and taking them through deployment.
Set and maintain the quality bar for owned models through rigorous evaluation and error analysis to ensure reliability for live operations.
Deploy models under real-time and compute constraints shared with the onboard perception system.
Partner with product, operations, safety, and vehicle software teams to translate requirements into model specifications, metrics, and validation plans.