Lead ML Inference Engineer, Advertising

Roku • Austin, TX

7d • Hybrid

About The Position

In this role, you will architect, design, and lead the development of a SOTA Inference platform that can handle Advertising-level low latencies, scale, throughput, and availability with optimizations that span across hardware, software, and models. We’re looking for a strong technical leader with deep experience in ML serving, high-performance computing, and industry standard frameworks - someone excited to mentor engineers, innovate at scale, and shape the future of machine learning at Roku.

Requirements

• M.S. or above in CS, ECE, or a related field
• 10+ years of experience in developing and deploying large-scale, distributed systems, with at least 5 years in a leadership or technical lead role
• Strong programming skills in high-performance languages
• Deep understanding of inference frameworks and ML system deployment
• Proven experience optimizing performance for large-scale machine learning systems, including a deep knowledge of SOTA model optimizations, hardware-software co-design, GPU acceleration, and HPC techniques
• Excellent communication and collaboration skills
• Experience leading teams working on high-throughput, low-latency ML serving systems
• Experience collaborating with and leading global, cross-functional teams

Nice To Haves

• Contributions to open-source ML or systems projects

Responsibilities

• Lead the design and development of a SOTA Inference platform
• Oversee the development of monitoring, observability, and other tooling to ensure system and model performance, reliability, and scalability of online inference services
• Identify and resolve system inefficiencies, performance bottlenecks, and reliability issues, ensuring optimized end-to-end performance
• Stay at the forefront of advancements in inference frameworks, ML hardware acceleration, and distributed systems, and incorporate innovations where and when they are impactful

Benefits

• Roku is committed to offering a diverse range of benefits as part of our compensation package to support our employees and their families.
• Our comprehensive benefits include global access to mental health and financial wellness support and resources.
• Local benefits include statutory and voluntary benefits which may include healthcare (medical, dental, and vision), life, accident, disability, commuter, and retirement options (401(k)/pension).
• Our employees can take time off work for vacation and other personal reasons to balance their evolving work and life needs.
• It's important to note that not every benefit is available in all locations or for every role. For details specific to your location, please consult with your recruiter.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level