Senior Machine Learning Engineer, Inference

AIDA Recruitment•Mountain View, CA

3d•Hybrid

About The Position

A fast-growing, well-funded AI company building real-time voice and LLM technology is seeking a Senior Machine Learning Engineer to join its inference team. The role owns inference optimization end-to-end, taking models from research through production deployment and addressing latency and throughput challenges for real-time AI products used at consumer scale. The team works on production inference engines, GPU efficiency, model acceleration, and distributed serving infrastructure.

Requirements

At least 5 years of experience as an ML inference or systems engineer, at senior level (Google L5, Meta E5, or equivalent).
Hands-on, engine-level ML inference optimization experience inside vLLM or TRT-LLM, beyond deployment alone.
Experience building backend or ML systems and serving models in production.
Systems programming and GPU optimization skills using C++, CUDA, Rust, or optimized Python.
Experience with model acceleration techniques, including quantization, distillation, speculative decoding, caching, continuous batching, and paged attention.
Distributed systems experience, including Kubernetes, Ray, load balancing, multi-GPU or multi-node inference, and high concurrency.
Based in the US, Canada, UK, Germany, Switzerland, or Serbia.
Excellent English communication skills.

Nice To Haves

A track record of fast promotions.
Public technical work, such as open-source inference engine contributions or technical write-ups.
A PhD in Computer Science, Physics, or Mathematics, or a degree from a top CS program.
Ability to commute to Mountain View, California.

Responsibilities

Own inference optimization end-to-end, from research models to production deployment.
Optimize latency and throughput for real-time voice and LLM inference at consumer scale.
Improve performance within inference engines such as vLLM, TRT-LLM, and SGLang.
Apply model acceleration techniques, including quantization, distillation, speculative decoding, caching, continuous batching, and paged attention.
Write high-performance code in C++, CUDA, Rust, or optimized Python for GPU workloads.
Build and scale distributed multi-GPU and multi-node serving using Kubernetes, Ray, and custom load balancing for high concurrency.
Collaborate with research and product teams to bring new models into production.