Senior Software Engineer, ML Serving Platform

DoorDash•San Francisco, WA

About The Position

DoorDash’s ML Serving Platform delivers tens of millions of predictions per second across DoorDash, Wolt, and Deliveroo, supporting search, recommendations, advertising, and logistics. The platform builds request routing, online feature retrieval, and CPU/GPU inference capabilities that enable ML and product teams to deploy and operate models independently. The Senior Software Engineer will lead core serving investments, make architectural decisions, write production code, and guide delivery with ML, product, and compute infrastructure partners. The first 6–12 months will focus on enabling new ranking and recommendation models, including generative approaches, in online production. The engineer will address bottlenecks with early product partners and turn solutions into reusable platform capabilities, enabling faster adoption while meeting model quality, latency, reliability, and cost targets.

Requirements

6+ years of software engineering experience designing, building, and operating production infrastructure or distributed systems.
Hands-on experience serving large or complex neural models online, with accountability for latency, availability, and resource efficiency.
Strong distributed systems fundamentals and proficiency in a backend or systems language such as Python, Go, Java, Kotlin, or C++.
Understanding of inference tradeoffs across batching, concurrency, caching, memory, and CPU/GPU utilization, with ability to diagnose production bottlenecks.
Experience operating services on Kubernetes or comparable infrastructure, including resource configuration, autoscaling, and failure recovery.
Experience leading substantial projects through ambiguity, partnering with fast-moving ML or product teams, and helping other engineers grow.
A degree in Computer Science or a related field, or equivalent practical experience.

Nice To Haves

Experience integrating or extending inference tools such as Triton Inference Server or PyTorch.
Experience with multi-GPU inference, model parallelism, model-aware caching, or accelerator scheduling.
Experience building model workflows, deployment tooling, or instrumentation with Metaflow, Helm, or OpenTelemetry.

Responsibilities

Lead projects from customer problem through design, implementation, and rollout.
Coordinate delivery across engineers, mentor peers, and help shape the team’s roadmap.
Improve inference performance using production traces, load tests, and CPU/GPU profiling.
Work with model owners to balance model quality, tail latency, throughput, and cost.
Partner with Compute on capacity, placement, and scaling, accounting for request complexity, cache state, GPU memory, and model startup behavior.
Build self-serve APIs and deployment workflows for managing models, runtime dependencies, features, and versioned state, with validation, safe rollout, and rollback.
Evaluate, integrate, and extend open source tools and vendor capabilities that advance model serving; decide when to build or adopt, and collaborate with maintainers and vendors.
Own production reliability through service objectives, observability, on-call, incident response, and follow-through.
Reduce recurring operational work through automation.