Software Engineer L4/L5, LLM Evaluation & Infrastructure, Machine Learning Platform

Netflix

93d • $100,000 - $720,000

About The Position

Netflix is one of the world's leading entertainment services, with over 300 million paid memberships in over 190 countries enjoying TV series, films and games across a wide variety of genres and languages. Members can play, pause and resume watching as much as they want, anytime, anywhere, and can change their plans at any time. Machine Learning/Artificial Intelligence powers innovation in all areas of the business, from helping members choose the right title for them through personalization, to better understanding our audience and our content slate, to optimizing our payment processing and other revenue-focused initiatives. Building highly scalable and differentiated ML infrastructure is key to accelerating this innovation. We are looking for a driven Software Engineer (L4/L5) to join our Machine Learning Platform (MLP) org. MLP’s charter is to maximize the business impact of all ML use cases at Netflix through highly reliable and flexible ML tooling and infrastructure that support personalization, studio algorithms, virtual production, growth intelligence, and content understanding. In this role, you will design and operate the systems that measure LLM quality, safety, and performance at scale—closing the loop from model development to production through rigorous, reproducible evaluation.

Requirements

• Experience in ML engineering on production systems dealing with training or inference of deep learning models.
• Proven track record of building and operating large-scale infrastructure for machine learning use cases.
• Experience with cloud computing providers, preferably AWS.
• Comfortable with ambiguity and working across multiple layers of the tech stack to execute on both 0-to-1 and 1-to-100 projects.
• Adopt and promote best practices in operations, including observability, logging, reporting, and on-call processes to ensure engineering excellence.
• Excellent written and verbal communication skills.
• Comfortable working in a team with peers and partners distributed across (US) geographies & time zones.

Nice To Haves

• End-to-end foundation-model lifecycle exposure: pre-train checks, post-train regression, and pre-launch gates.
• Built or contributed to an evaluation platform at scale with strong SLIs/SLOs.
• Experience building evaluation data pipelines with provenance and governance.
• Platform mindset: craft usable APIs/UX for modeling teams.
• Bonus signals across the broader stack: experience with reinforcement learning, agent modeling, AI alignment, distributed training, vector search/feature stores, routing/safety middleware for serving, and cost/perf tuning.

Responsibilities

• Build the evaluation platform that runs large-scale LLM eval suites across modalities and tasks, integrating with batch/online inference and experiment tracking to deliver reliable, reproducible metrics.
• Operationalize benchmark coverage alongside Netflix-specific task suites and user-journey-grounded prompts; automate result collection, statistical analysis, and drift detection.
• Develop high-quality synthetic data and labeling pipelines to expand coverage, reduce bias, and continuously refresh eval corpora; codify data provenance and sampling policies.
• Partner deeply with model developers and platform teams to co-design APIs for submitting eval jobs, adding new tasks/metrics, and defining SLO-like quality thresholds that unblock launches while preventing regressions.
• Contribute beyond evaluation across the GenAI/FM stack when needed, including research workflows, inference foundations, and observability for the whole loop.

Benefits

• Comprehensive Health Plans
• Mental Health support
• 401(k) Retirement Plan with employer match
• Stock Option Program
• Disability Programs
• Health Savings and Flexible Spending Accounts
• Family-forming benefits
• Life and Serious Injury Benefits
• Paid leave of absence programs
• 35 days annually for paid time off for full-time hourly employees
• Flexible time off for full-time salaried employees

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Number of Employees: 5,001-10,000 employees