Senior Machine Learning Engineer, Inference

Inworld•Mountain View, CA

12h•$250,000 - $300,000•Onsite

About The Position

Inworld is hiring a Senior Machine Learning Engineer focused on inference to build and optimize real-time AI serving systems for consumer-facing applications. The role involves taking models from research through containerization and serving optimization to reliable production operation, with performance, latency, and reliability treated as core product features. The position offers unclear, technically challenging problems and expects engineers to define the problem, investigate solutions through benchmarks or prototypes, and deliver stable systems. The team works in person from its Mountain View office and offers relocation assistance.

Requirements

Strong ability to learn quickly and thrive in ambiguity.
Deep understanding of modern inference-serving frameworks and techniques, such as vLLM or TRT-LLM.
Hands-on experience with model acceleration techniques such as quantization, distillation, caching strategies, continuous batching, paged attention, or speculative decoding.
Proficiency in C++, CUDA, Rust, or highly optimized Python; ability to profile code and optimize performance on NVIDIA GPUs.
Experience with distributed systems and scaling, including Kubernetes, Ray, custom load balancing, multi-GPU or multi-node inference, and handling thousands of concurrent connections.
Ability to take ownership of the full model-serving lifecycle, from research handoff through production reliability.
PhD in Computer Science, Physics, or Mathematics, or equivalent practical experience building backend or machine-learning systems.

Nice To Haves

Non-trivial systems programming projects.
Open-source contributions to major inference engines.
In-depth technical write-ups.

Responsibilities

Take models from the research team through containerization, serving optimization, and reliable production deployment.
Optimize model inference for performance, latency, and throughput.
Build and operate scalable, reliable inference systems for high-volume workloads and thousands of concurrent connections.
Investigate unclear technical problems by designing benchmarks or prototypes and use findings to guide solutions.
Question and improve architectures to address core latency and throughput challenges.
Contribute to impactful engineering work, with opportunities to share work and make open-source contributions.