Engineering Manager, Online Serving and Inference, AI Platform
About The Position
Netflix’s Artificial Intelligence Platform (AIP) organization builds scalable AI/ML infrastructure to support AI practitioners across the business. The Online Serving and Inference team develops and maintains infrastructure for large-scale online model serving and inference, including common frameworks for developing and deploying AI/ML models in Netflix infrastructure. The Engineering Manager will lead development of a next-generation serving and inference platform. The platform will help practitioners across content, studio, consumer, ads, and games package, deploy, and run inference workflows for thousands of large-scale models, including LLMs, computer vision models, and foundation models at multiple lifecycle stages. The role partners across engineering, product management, machine learning, and data teams to build scalable, robust systems that support evolving AI needs. The position combines technical platform leadership with people management. The manager will guide architecture and execution, develop an incremental roadmap and adoption plan, maintain existing offerings while building the next-generation platform, communicate with stakeholders and senior leadership, and hire and develop a diverse engineering team. The role manages a hybrid team whose members and partners are distributed across US geographies and time zones.
Requirements
- 10+ years of software engineering experience and 3+ years of management experience.
- Experience leading teams responsible for building high-traffic distributed systems and ML infrastructure.
- Experience with containerization and orchestration technologies, such as Kubernetes, supporting data preparation, processing, and inference for large-scale AI/ML models.
- Industry experience with GPU-based model inference and serving optimization, including batching, latency, and cost per request, in large-scale computing environments.
- Experience with modern foundation-model serving and inference stacks, such as vLLM, Nvidia Triton, TensorRT LLM, or SGLang.
- Strong technical acumen and ability to act as a credible technical advisor, set and enforce high standards for code and system design, and mentor engineers.
- Passion for translating ML practitioners’ needs into infrastructure offerings with automated and self-serve capabilities.
- Strong communication and collaboration skills, including the ability to build relationships with internal customers and external partners.
- Demonstrated ability to develop, drive, and execute a technical vision and roadmap.
- Track record of attracting talent and leading and growing high-performing, diverse, senior- and staff-level engineers in a fast-paced environment.
- Experience managing a hybrid team with partners and team members distributed across US geographies and time zones.
Responsibilities
- Partner with Applied Research and backend application teams to understand their needs, gather platform requirements, and iterate on those requirements.
- Drive the architecture, design, build-versus-buy evaluation, and execution of the online serving and inference platform.
- Design for extensibility and scale as AI requirements evolve.
- Build an incremental-delivery roadmap, define success metrics, align on migration goals, and drive platform adoption.
- Communicate progress to stakeholders, customers, and senior leadership.
- Maintain existing platform offerings while balancing immediate needs with development of the next-generation platform.
- Hire and grow engineers and foster an inclusive team culture.
- Provide technical guidance, set and enforce a high-quality bar for code and system design, and mentor the team.
- Develop, drive, and execute a technical vision and roadmap.
- Lead and grow high-performing teams of senior and staff-level engineers.
Benefits
- Health plans
- Mental health support
- 401(k) retirement plan with employer match
- Stock option program
- Disability programs
- Health Savings and Flexible Spending Accounts
- Family-forming benefits
- Life and Serious Injury Benefits
- Paid leave of absence programs
- Flexible time off for full-time salaried employees
- Full-time hourly employees accrue 35 days annually for vacation, holidays, and sick paid time off
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Manager
Education Level
No Education Listed
