SWE, Data Ingestion

Wayve • Sunnyvale, CA

11h • Hybrid

About The Position

Wayve is seeking a Data Ingestion Engineer to build and strengthen the data foundations for their self-driving technology. This role is crucial as Wayve trains vehicles to drive from data, making data ingestion a key part of their learning system. The faster and more intelligently real-world driving data can be processed, the quicker Wayve can improve its models and bring embodied AI to the world. This is a hands-on, permanent role for an engineer who enjoys solving practical, high-impact problems at scale. The engineer will ensure ingestion pipelines run smoothly, unblock critical data flows, and contribute to the long-term evolution of systems supporting annotation, data science, and model training/evaluation. Wayve's data platform handles over 500,000 hours of driving data (100s of PBs), and the ingestion systems need to be robust, efficient, and cost-effective to support growing ADAS and autonomy work. A single bad data segment can disrupt pipelines and slow down downstream teams, highlighting the direct impact of this role on Wayve's AI learning speed.

Requirements

• Strong production experience with Apache Spark.
• Strong Python engineering experience.
• Experience building, debugging, or operating large-scale data-ingestion, ETL, or data-processing pipelines.
• Experience with distributed data-processing systems.
• Ability to optimize jobs for throughput, compute efficiency, and reliability.
• Experience debugging production pipeline failures.
• Comfort working with messy, corrupt, incomplete, or inconsistent data.
• Understanding of orchestration across multi-step pipelines and downstream dependencies.
• Ability to work independently in a fast-moving, highly technical environment.
• A practical, delivery-focused mindset with a focus on continuous improvement.
• Experience working at significant data scale, ideally PB-scale or similarly high-throughput environments.

Nice To Haves

• Experience in one or more of the following areas would be a strong advantage: Airflow, Flyte, Databricks Workflows or similar orchestration tooling.
• Databricks, Delta Lake or Delta tables.
• Scala or Java, especially in Spark-based environments.
• Queue-based processing, retry handling, and priority data workflows.
• High-throughput batch data-processing systems.
• Production systems with many data producers, consumers, or external data sources.
• Handling third-party, partner, or supplier data with inconsistent formats and quality issues.
• Automotive, robotics, autonomy, mapping, ML data platforms, or embodied AI environments.
• Cost optimization for compute- and storage-heavy data platforms.
• High-performance engineering experience from domains such as trading, where it includes relevant distributed-systems or throughput-focused work.

Responsibilities

• Improve the reliability, efficiency, and throughput of pipelines that move real-world driving data through Wayve.
• Debug and resolve failing or blocked ingestion pipelines.
• Investigate issues caused by corrupt, malformed, or unexpected data.
• Design and implement more resilient pipelines to prevent individual bad data segments from blocking wider workflows.
• Improve handling of varied data formats from partners, suppliers, and third-party sources.
• Support orchestration across multi-step ingestion workflows, including dependencies, retries, and queue management.
• Optimize Spark jobs and data-processing pipelines for throughput, compute efficiency, and reliability.
• Reduce operational toil around failed jobs, stalled pipelines, and manual interventions.
• Work on high-volume batch-processing systems where throughput, reliability, and cost are critical.
• Help prioritize and unblock important datasets for downstream annotation, data science, and model training teams.
• Partner with engineers across Data Platform and downstream teams to deliver immediate improvements and scalable long-term solutions.
• Contribute to the technical direction, maintainability, and operational excellence of Wayve’s ingestion platform.

Benefits

• Competitive equity package
• Hybrid working policy
• Commitment to creating a diverse, fair, and respectful culture that is inclusive of everyone
• Accommodations or adjustments to participate fully in the interview process

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed