Staff AI Engineer

Zoom • Seattle, WA

6d • $206,600 - $451,800

About The Position

The Staff AI Engineer will design, implement, and own production-scale inference systems serving Zoom AI models across real-time communication, computer vision, and language workloads. The role is hands-on with kernel-level optimization, inference framework internals, and production serving infrastructure, partnering with research and platform teams to improve latency, throughput, and cost.

The position is part of Zoom's AI Infrastructure team, which builds core systems for model training, deployment, and inference across Zoom products and services. The engineer will build and optimize serving systems for large transformer and multimodal models, contribute to inference frameworks, write and profile GPU kernels, and own end-to-end deployment and operational performance.

Requirements

• Bachelor's or Master's degree in Computer Science, Electrical Engineering, or a related technical field, or equivalent practical experience.
• At least 5 years of software engineering experience, with significant experience in production-depth inference systems or ML infrastructure.
• Hands-on serving experience with at least one major inference framework: vLLM, TensorRT-LLM, SGLang, or ONNX Runtime.
• GPU programming experience, including CUDA kernel development, memory optimization, and profiling with Nsight or equivalent tools.
• Production experience serving LLMs or large vision models, including ownership of latency SLOs, debugging throughput regressions, and delivering impactful optimizations.
• Depth in at least two of the following: speculative decoding, continuous batching, KV cache design, quantization pipelines, or prefill/decode disaggregation.
• Strong systems skills in Python and C++, including the ability to read and modify framework internals.

Nice To Haves

• Advanced degree (Master's or PhD) in a relevant technical field.
• Experience with MoE models or deployments of models with 100B+ parameters.
• Familiarity with disaggregated serving architectures or multi-node inference.
• Background in compiler-level optimization, such as XLA or Triton.

Responsibilities

• Design and build high-performance inference serving systems for large-scale transformer and multimodal models, including 100B+ parameter and MoE architectures.
• Implement and tune inference optimizations, including speculative decoding, continuous batching, KV cache management, prefill/decode disaggregation, and INT4/INT8/FP8 quantization.
• Contribute to and customize inference frameworks such as vLLM, TensorRT-LLM, SGLang, or equivalent for Zoom's production requirements.
• Write and profile CUDA kernels and custom operations when framework-level optimization is insufficient.
• Own end-to-end deployment, from model packaging and serving API design to latency SLO monitoring and incident response.
• Partner with research teams to translate model architecture changes into inference-efficient implementations.
• Drive technical design and establish inference engineering practices across the team.

Benefits

• The benefits program offers a variety of perks, benefits, and options intended to support physical, mental, emotional, and financial health, work-life balance, and community involvement.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior