Senior Software Engineer, ML Infrastructure

Gatik AI•Santa Clara, CA

1d•$180,000 - $260,000

About The Position

Gatik is an autonomous freight company whose Level 4 Gatik Driver™ moves freight for retailers, grocers, and consumer brands across the United States and Canada. The Scalable ML team builds the data-generation, curation, orchestration, and distributed-training infrastructure that enables machine-learning developers to work with real-world fleet data and simulated driving scenarios. The team collaborates with ML Platform, Jobs Platform, Cloud Platform, High Performance Computing, autonomy, simulation, safety, and data teams. As a Senior or Staff engineer on Scalable ML, you will design and build systems that transform raw driving data into high-quality training and evaluation datasets, run training and evaluation reliably at scale, and provide engineers with metrics and tools for release decisions supporting driverless operations. Staff-level engineers will also set multi-quarter technical direction and serve as a technical bridge across teams. The role includes building reproducible data pipelines; curating balanced, high-quality datasets; scaling distributed training, inference, and evaluation; establishing MLOps practices; defining and validating rigorous model and release metrics; conducting experiments and communicating results; creating tools for exploring and comparing model results; shaping shared compute infrastructure; and raising engineering quality and reliability. Staff engineers additionally lead technical strategy, represent Scalable ML in architecture discussions, and mentor engineers.

Requirements

Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, Robotics, or a related field and 5+ years of relevant industry experience, or a Master’s degree and 3+ years.
Strong Python software engineering skills, including designing modular, maintainable, high-performance, well-tested systems in a shared codebase.
Experience building end-to-end ML systems from data pipelines and dataset curation through training, evaluation, and production, using tools such as SQL, pandas, PyTorch, or scikit-learn.
Hands-on production experience with Ray (Ray Core, Ray Data, Ray Train, and/or Ray Tune), including running and debugging Ray workloads at scale.
Deep experience with PyTorch and PyTorch Lightning for large-scale training, including distributed strategies such as DDP or FSDP, mixed precision, and checkpointing.
Experience designing and operating scalable ML infrastructure, such as distributed training and evaluation pipelines, experiment tracking, model serving, or large-scale distributed jobs.
Familiarity with shared-compute environments, including GPU scheduling and quota, job orchestration, and multi-tenant clusters.
Solid grounding in statistics and experimental design, including selecting evaluation strategies, metrics, and thresholds to support real-world decisions.
Clear communication skills and ability to work effectively across software, ML, and autonomy teams.
Staff candidates typically have 8+ years of experience, including experience setting technical direction beyond a single team.

Responsibilities

Design and maintain reproducible pipelines that discover, label, and organize fleet and simulation driving scenarios, including versioning, provenance, train/validation/holdout splits, and data contracts.
Build balanced training and evaluation datasets that cover truck operating scenarios across routes, weather, and traffic conditions; analyze dataset coverage, composition, and label quality.
Build distributed training, inference, and evaluation pipelines with robust data loading, checkpointing, and experiment tracking for multi-tenant GPU clusters.
Implement model artifact management, experiment tracking, and automated model testing and deployment practices for adoption across Gatik.
Define, implement, and validate model and release metrics, including precision, recall, F1, ROC AUC, calibration, and task-specific metrics; analyze behavior across releases and scenario types.
Set up baselines, ablations, and holdout comparisons, quantify uncertainty, and communicate results to support autonomy team decisions.
Develop dashboards and services for comparing software versions, exploring scenario-level results, visualizing model outputs, and gathering reviewer feedback.
Partner with platform and HPC engineers on GPU scheduling and quota, job orchestration, storage, and compute cost.
Design clear interfaces between data, modeling, and product layers; promote testing, type checking, code review, and fast CI; lead incident root-cause analysis and prevention, including cross-team incidents.
At Staff level, define and drive multi-quarter technical strategy for data and training infrastructure, represent Scalable ML in architecture discussions, and mentor engineers.