AI Engineer

Zoom • Seattle, WA

10d

About The Position

Join Zoom’s AI Infrastructure team, which builds core systems for high-performance AI across Zoom products and services, supporting model training, deployment, and inference at scale. The role focuses on building and optimizing production inference systems for Zoom’s AI models, working with senior inference engineers and research and platform teams to profile serving workloads, identify latency, throughput, and cost bottlenecks, and implement improvements. The work involves inference frameworks, GPU profiling, and serving infrastructure.

The engineer will build and maintain components of the inference serving stack for transformer and multimodal models, including large and Mixture-of-Experts architectures. They will implement inference optimizations, extend frameworks to support new models and production needs, develop performance tooling, collaborate with researchers to bring architectures into production, and contribute to on-call and production support with guidance from senior engineers.

Requirements

• A degree in Computer Science, Computer Engineering, or a related field, or equivalent practical experience; an MS or PhD is preferred.
• 0–3 years of software engineering experience; internships, research, and open-source work in ML systems count.
• Hands-on experience serving and modifying at least one inference framework, such as vLLM, SGLang, TensorRT-LLM, or ONNX Runtime.
• Working knowledge of LLM inference on GPUs, including memory bandwidth, KV cache, batching, and communication across GPUs.
• Experience profiling and debugging GPU performance, with evidence of a measured optimization that was shipped.
• Depth in at least one of speculative decoding, continuous batching, KV cache design, quantization, or distributed or multi-GPU serving.
• Strong programming skills in Python and C++, and comfort reading and changing large framework codebases.

Nice To Haves

• Merged contributions to vLLM, SGLang, TensorRT-LLM, PyTorch, or similar projects.
• CUDA, Triton, or CUTLASS kernel development experience.
• Experience serving MoE or 100B+ parameter models, or multi-node inference using NCCL or RDMA/RoCE.
• Graduate coursework or research in ML systems, compilers, or high-performance computing.

Responsibilities

• Build, tune, and maintain components of Zoom’s inference serving stack for transformer and multimodal models, including large and Mixture-of-Experts (MoE) architectures.
• Implement and evaluate inference optimizations, including speculative decoding, continuous batching, KV cache management, parallelism layouts, and INT4/INT8/FP8 quantization.
• Extend and customize inference frameworks such as vLLM, SGLang, TensorRT-LLM, or equivalent to support new models and Zoom production requirements.
• Profile GPU workloads with Nsight Systems or equivalent tools, identify causes of performance regressions, and validate improvements with reproducible benchmarks.
• Build and improve internal tooling for benchmarking, performance modeling, and serving configuration.
• Partner with researchers to bring new model architectures into production efficiently.
• Contribute to on-call and production support for the services built, with guidance from senior engineers.

Benefits

• Zoom describes a benefits program with perks, benefits, and options intended to support employees’ physical, mental, emotional, and financial health, work-life balance, and community involvement.
• The compensation philosophy considers base salary, bonus, and equity value.

What This Job Offers

• Job Type: Full-time
• Career Level: Entry Level