Staff Engineer, Autonomous Driving Data Platform & Curation

Cariad, Inc.•Mountain View, CA

3h•$161,710 - $234,325

About The Position

We are CARIAD, an automotive software development team with the Volkswagen Group. Our mission is to make the automotive experience safer, more sustainable, more comfortable, more digital, and more fun. To achieve that we are building the leading tech stack for the automotive industry and creating a unified software platform for over 10 million new vehicles per year. We’re looking for talented, digital minds like you to help us create code that moves the world. Together with you, we’ll build outstanding digital experiences and products for all Volkswagen Group brands that will transform mobility. Join us as we shape the future of the car and everyone around it. Role Summary: The Staff Engineer, Autonomous Driving Data Platform & Curation is a hands-on staff-level individual contributor who owns the path from raw multimodal vehicle data to reliable, versioned, and model-ready datasets. The ideal candidate combines production data or ML systems experience with practical knowledge of autonomous-driving data and can lead ingestion, schema, storage, query, curation, quality, and delivery. This engineer designs and operates datasets containing more than 10 million records or samples, evaluates formats such as Parquet and Lance, and enables efficient filtering, slicing, random access, and sequential retrieval. The role also advances data-quality monitoring, statistical and out-of-distribution detection, rule-based and model-based tagging, model-in-the-loop and human-in-the-loop labeling, and future data preparation for imitation learning and reinforcement learning.

Requirements

Deep production experience with Parquet and/or Lance, including dataset migration, indexing, compaction, schema evolution, version maintenance, cloud storage, random access, or training-loader optimization.
Experience with large-scale multimodal autonomous-driving data from cameras, LiDAR, radar, maps, localization, vehicle state, simulation, or other sensor-rich robotic systems.
Experience building auto-labeling, active-learning, model-in-the-loop, or human-in-the-loop systems, including confidence calibration, annotation routing, review workflows, audit sampling, label agreement, and annotation QA.
Strong statistical knowledge for quality monitoring, distribution comparison, drift measurement, anomaly detection, and out-of-distribution detection.
Experience using rule-based logic, geometry and metadata heuristics, embeddings, semantic search, VLMs, classifiers, or other learned methods to tag data, assess quality, mine edge cases, detect hard examples, or prioritize review.
Experience connecting dataset metrics, composition, label quality, drift, cost, and downstream model performance through dashboards and evaluation.
Experience optimizing distributed ML data loading, caching, sampling, preprocessing, and the data-to-GPU path.
Experience preparing sequential or trajectory datasets for imitation learning, offline reinforcement learning, preference or ranking data, reward modeling, or policy evaluation, including observations, actions, rewards, policy provenance, and replay semantics.
Familiarity with data governance, privacy, retention, redaction, access control, auditability, and security requirements for fleet-collected sensor data.
Demonstrated ownership of production data pipelines and large datasets, including systems with more than 10 million records or samples or comparable multi-terabyte scale.
Strong dataset and storage foundation, including schema design, columnar storage, partitioning, file and fragment sizing, compression, predicate pushdown, projection, indexing, compaction, schema evolution, versioning, lineage, and reproducibility.
Hands-on experience with Parquet, Lance, Apache Arrow, or a comparable analytical dataset technology, with the ability to reason about format selection and benchmark storage, query, update, and model-loading trade-offs.
Strong Python and SQL skills with production software-engineering practices, including testing, code review, version control, profiling, debugging, maintainable APIs, and operational documentation.
Experience building distributed or parallel data pipelines and orchestration using technologies such as Ray, Spark, Beam, Dask, Airflow, Dagster, or comparable systems.
Experience with local or cloud object storage, such as S3 or GCS, and practical understanding of throughput, request patterns, caching, data movement, reliability, access control, and cost.
Basic autonomous-driving or robotics data knowledge, including temporal sensor data, camera and other sensor modalities, timestamps and synchronization, calibration, ego and vehicle state, coordinate frames, clips, trajectories, scenarios, and annotations.
Ability to design automated data validation and observability and translate ML training, sampling, evaluation, and failure-analysis needs into reliable data products.
8+ years of experience in applied machine learning or deep learning
4+ years of experience reinforcement learning, computer vision, or AD/ADAS systems.
Master’s in Computer Science, Robotics, Electrical Engineering, Applied Mathematics, or a related field

Nice To Haves

Deep production experience with Parquet and/or Lance, including dataset migration, indexing, compaction, schema evolution, version maintenance, cloud storage, random access, or training-loader optimization.
Experience with large-scale multimodal autonomous-driving data from cameras, LiDAR, radar, maps, localization, vehicle state, simulation, or other sensor-rich robotic systems.
Experience building auto-labeling, active-learning, model-in-the-loop, or human-in-the-loop systems, including confidence calibration, annotation routing, review workflows, audit sampling, label agreement, and annotation QA.
Strong statistical knowledge for quality monitoring, distribution comparison, drift measurement, anomaly detection, and out-of-distribution detection.
Experience using rule-based logic, geometry and metadata heuristics, embeddings, semantic search, VLMs, classifiers, or other learned methods to tag data, assess quality, mine edge cases, detect hard examples, or prioritize review.
Experience connecting dataset metrics, composition, label quality, drift, cost, and downstream model performance through dashboards and evaluation.
Experience optimizing distributed ML data loading, caching, sampling, preprocessing, and the data-to-GPU path.
Experience preparing sequential or trajectory datasets for imitation learning, offline reinforcement learning, preference or ranking data, reward modeling, or policy evaluation, including observations, actions, rewards, policy provenance, and replay semantics.
Familiarity with data governance, privacy, retention, redaction, access control, auditability, and security requirements for fleet-collected sensor data.
Strong candidates with equivalent industry experience will be considered
PhD in Computer Science, Robotics, Electrical Engineering, Applied Mathematics, or a related field

Responsibilities

Design canonical representations for drives, scenarios, clips, frames, trajectories, sensor references, vehicle state, map context, labels, predictions, and dataset manifests.
Build and maintain validated, versioned datasets containing more than 10 million records or samples on local or cloud object storage.
Evaluate Parquet, Apache Arrow, Lance, and related technologies; own partitioning, indexing, file sizing, compaction, schema evolution, lineage, and reproducibility.
Optimize filtering, projection, joins, scenario slicing, random sampling, shuffling, sequential retrieval, and model data-loading performance.
Measure and improve ingestion throughput, query latency, training throughput, storage utilization, reliability, and cost per usable sample.
Work effectively with synchronized camera and other sensor data, ego state, localization, calibration, coordinate frames, map context, control actions, clips, and trajectories.
Translate perception, planning, VLA, and evaluation needs into schemas, searchable attributes, scenario definitions, sampling strategies, and reproducible dataset splits.
Build reliable batch or distributed pipelines for ingestion, transformation, enrichment, validation, cataloging, and publication, including retries, backfills, idempotency, and observability.
Enable self-service discovery and composition for lane keeping, lane changes, long-tail scenarios, hard examples, balanced datasets, and leakage-resistant train, validation, and test splits.
Define automated quality gates, dashboards, and alerts for completeness, validity, freshness, duplication, synchronization, calibration, corruption, label integrity, coverage, balance, and cost.
Apply statistics, sampling, and distribution comparisons to detect drift, anomalies, underrepresented conditions, and out-of distribution data.
Use deterministic checks, heuristic rules, geometry and metadata queries, embeddings, VLMs, and learned models to assess quality and create searchable scenario tags.
Quarantine suspicious data and lead root-cause analysis across collection, synchronization, schema, transformation, storage, annotation, sampling, and model-consumer failures.
Design model-in-the-loop and human-in-the-loop labeling workflows with confidence thresholds, review routing, audit sampling, disagreement handling, and label provenance.
Close the loop from model failures and edge cases through selection, annotation, quality review, dataset publication, training, and evaluation.
Evaluate auto-labeled and synthetic data using label quality, coverage, distributional impact, and downstream model performance.
Prepare replayable trajectory data for imitation learning and offline RL, including observations, actions, timestamps, policy versions, interventions, rewards, termination conditions, and alignment checks.
Serve as the staff-level owner and escalation point for dataset architecture, storage and query performance, data quality, reliability, reproducibility, and cost.
Align vehicle collection, autonomy metadata, model interfaces, annotation, storage, governance, and training requirements across partner teams.
Communicate risks, trade-offs, decisions, and recovery plans with clear evidence; maintain traceable schemas, data contracts, lineage, operating procedures, and incident records.
Mentor engineers and strengthen design reviews, testing, observability, reproducibility, cost awareness, and data engineering standards.