L7): AI Platform Model Serving

Netflix•Remote, OR

1d•$890,000 - $1,690,000

About The Position

Netflix is rebuilding its AI/ML model-serving platform and infrastructure to serve predictions across real-time, asynchronous, and agentic workloads. The platform supports production clusters ranging from classical models to foundation models, with latencies from milliseconds to LLM calls, and powers member-facing experiences and internal AI productivity tools. The new platform is being designed from first principles to unify in-house inference infrastructure, third-party model hosting, and external LLM providers under a consistent serving, routing, and experimentation layer. This role is central to Netflix’s GenAI and Foundation Model serving strategy, with responsibility for technical direction and architecture in a growing engineering organization. The work includes improving GPU capacity provisioning and utilization, including reusing off-peak capacity for internal open-weight model workloads, and addressing efficiency challenges that affect inference cost and capacity. Areas include disaggregated serving, intelligent model and provider routing, speculative decoding, prompt caching, multi-GPU inference, and durable, resumable execution for long-running agentic workloads. The engineer will guide platform technology and architecture for AI capabilities used in Netflix products at global scale, balancing performance, cost, reliability, scalability, maintainability, and business priorities. The position calls for deep expertise in distributed systems, high-performance RPC, GPU/CPU inference optimization, control planes, experimentation infrastructure, and modern LLM serving stacks.

Requirements

BS, MS, or PhD in Computer Science, Applied Math, Engineering, or a related field.
12+ years of relevant experience in AI/ML infrastructure and distributed systems.
Experience with high-traffic distributed services and AI/ML model-inference infrastructure, including high availability, security, and performance.
Experience building scalable model-serving solutions for foundation models and reducing inference latency and costs.
Familiarity with deploying AI/ML models using platforms such as NVIDIA Triton Inference Server, vLLM, or SGLang, and scaling them with Kubernetes.
Experience with public cloud infrastructure providers such as AWS, Azure, or GCP.
Ability to influence architectural decisions and align multiple teams without formal authority.
Experience shaping long-term technical roadmaps and partnering across product, operations, and executive stakeholders.
Clear communication skills, including explaining technical concepts to non-technical stakeholders and producing documentation, ADRs, and proposals.

Nice To Haves

Open-source contributions in AI/ML, distributed systems, or infrastructure.
Hands-on experience building AI models and exploring optimization techniques such as quantization, model compression, or model pruning.
Proficiency leveraging AI coding tools in software development.
Experience with model gateways and intelligent routing to select models based on request context.
Exposure to multi-adapter model serving, disaggregated prefill/decode, or large-scale KV cache management.
Experience with evaluation and benchmarking tools for LLMs and AI agents.
Knowledge of GPU topology, NVIDIA CUDA programming, and machine-learning compilers.

Responsibilities

Lead technical architecture and direction for the model-serving platform and its engineering organization.
Shape long-term technical roadmaps, typically two to three years ahead, while ensuring scalability, maintainability, and business alignment.
Guide architectural decisions and alignment across multiple teams through influence without authority.
Build and evolve infrastructure for high-traffic AI/ML inference services, with attention to availability, security, and performance.
Develop scalable foundation-model serving solutions for generative and predictive AI applications, reducing latency and cost.
Improve GPU capacity provisioning and utilization, including repurposing off-peak capacity for internal workloads.
Address inference efficiency challenges through areas such as disaggregated serving, intelligent routing, speculative decoding, prompt caching, and multi-GPU inference.
Build durable, resumable execution capabilities for long-running agentic workloads.
Partner with product managers, operations, and executive stakeholders to balance technical trade-offs with business priorities.
Communicate technical concepts to non-technical stakeholders and produce documentation, architecture decision records, and proposals.

Benefits

Health plans
Mental health support
401(k) retirement plan with employer match
Stock option program
Disability programs
Health Savings and Flexible Spending Accounts
Family-forming benefits
Life and Serious Injury Benefits
Paid leave of absence programs
Full-time hourly employees accrue 35 days annually for vacation, holidays, and sick paid time off
Full-time salaried employees are immediately entitled to flexible time off