Member of Technical Staff — Model Optimization and Inference (New Grad)

Nuance Labs•Seattle, WA

2d•$200,000 - $300,000•Onsite

About The Position

Nuance Labs is building photorealistic, real-time AI avatars with emotional intelligence: a full-duplex audiovisual system that can listen, speak, react, interrupt, and respond like a real person. We're a Series A company ($60M raised) backed by Lightspeed, Accel, South Park Commons, NVentures, and Define Ventures, with PhDs from MIT, UW, Oxford, CMU, and Johns Hopkins, and industry experience from Apple, Meta, Amazon AGI, and Discord. The team is small, the work is real, and the problems are unsolved. Most conversational AI avatars today are hacks — a face slapped on a speech-to-speech pipeline, stuck in the uncanny valley: emotionless, mechanical, one-turn-at-a-time. Current systems take 2–5 seconds to respond; natural conversation requires sub-500ms. That's a 10x improvement, and it demands rethinking the entire stack. That rethinking starts with full-duplex: an AI that listens and speaks simultaneously, perceives emotion in real time, and responds with a face that actually reflects it. It's an extremely hard problem, and we're developing foundation models designed for it from the ground up. We can train a great model. The next problem is making it fast enough to actually use in a real-time conversation — and that gap is enormous. A model that responds in 3 seconds is a demo. A model that responds in under 500ms is a product. We’re looking for someone who’s excited about taking trained models and squeezing every last millisecond out of them. You understand — or want to deeply understand — the full stack from model weights to serving infrastructure: quantization, KV cache optimization, kernel-level acceleration, batching strategies. You’ve worked with vLLM, SGLang, or similar frameworks (through coursework, research, internships, or open-source) and have opinions about where they fall short. This posting is aimed at early-career engineers finishing or recently finished with a BS, MS, or PhD. We don’t require a PhD — we care about systems intuition, engineering chops, and the appetite to go deep. Our stack is more complex than a standard LLM deployment: we’re serving a full-duplex multimodal system that must satisfy strict real-time latency constraints. There’s a lot of unsolved optimization work here, and we want someone who finds that genuinely exciting and is ready to grow fast alongside people who’ve built these systems before.

Requirements

BS, MS, or PhD in CS, ML, or a related field — completed or in the final stretch
Strong fundamentals in LLM inference or ML systems — KV caching, memory layout, attention kernels, batching, or serving — picked up through coursework, research, internships, or open-source. You don’t need to have shipped at production scale yet; you do need to learn fast and go deep.
Exposure to inference serving frameworks (vLLM, SGLang, TensorRT-LLM, or similar) — even at a research or hobby level
Strong Python and PyTorch skills; familiarity with CUDA or Triton is a significant plus
A systematic approach to profiling and optimization — you measure first, then optimize
Curiosity about diffusion inference, speculative decoding, quantization, or other inference-time acceleration techniques

Nice To Haves

Internship or research experience with LLM inference, ML systems, or model serving
Contributions to open-source inference frameworks (vLLM, SGLang, TensorRT-LLM, etc.)
CUDA / Triton kernel work, even at a research or hobby scale
Publications or research projects in MLSys, model compression, or inference optimization
Familiarity with multimodal or streaming inference architectures
Experience with hard latency SLAs in any real-time system

Responsibilities

Contribute to end-to-end inference optimization across our model stack — LLMs, audio models, and diffusion-based components
Implement and tune KV cache strategies for long-context conversations, including eviction policies, compression, and memory-efficient attention
Work with inference serving frameworks (vLLM, SGLang, TensorRT-LLM, etc.) and extend them for our specific workloads
Profile and benchmark end-to-end latency and throughput; identify and systematically eliminate bottlenecks
Build internal tooling that makes optimization work faster and more rigorous — profiling viewers, end-to-end inference test harnesses, and other infrastructure that helps the team move quickly
Accelerate diffusion model inference — consistency models, step distillation, caching strategies, and custom kernel optimizations
Apply quantization techniques (INT8, INT4, GPTQ, AWQ, and beyond) to reduce memory footprint and increase throughput without meaningfully degrading quality
Work closely with research and infrastructure to ensure new models ship with optimized serving from day one