Senior Research Engineer

AssemblyAI•New York, NY

10d•$270,000 - $310,000•Remote

About The Position

AssemblyAI is seeking a Senior Research Engineer to join their Research team. This role focuses on developing and improving the systems behind large-scale distributed training, data processing, and inference for Voice AI models. The primary goal is to accelerate the pace of model development and product improvement by enhancing the speed and reliability of experimental pipelines. The ideal candidate will possess a deep understanding of modern deep learning systems, with expertise in JAX, TPUs, layer-level optimization, large-scale distributed training, streaming and low-latency inference, inference compilers, and advanced parallelization techniques. This is a cross-functional role requiring close collaboration with researchers, infrastructure, and production engineering teams to troubleshoot and resolve issues end-to-end. The role is embedded within the Research team.

Requirements

Expert-level proficiency with JAX and TPUs, including the surrounding ecosystem (Flax, Optax, the XLA compilation pipeline).
Measurement discipline. You define what success looks like before you start, you stay skeptical of your own results until they hold up, and you treat an unexplained improvement as a problem rather than a win.
Appetite for the whole pipeline. Your core strength might be JAX and TPU performance, but when a customer issue traces back to a data problem or an evaluation blind spot, you want to go find it yourself. The people who do well here went deep in one area first, then kept expanding outward.
Strong experience optimizing inference systems for production, ideally with LLMs or speech models.
Deep understanding of distributed training at scale, modern deep learning systems, and ML infrastructure best practices.
Familiarity with modern inference optimization techniques: continuous batching, KV-cache management, sharding strategies, quantization.
Enthusiasm for refactoring and improving existing systems — you thrive on making products and code faster and better.
Strong Python skills; C++ or Rust experience for kernel-level work is a plus.
Excellent communication and a collaborative mindset — you can clearly explain complex tradeoffs and prioritize high-impact work.

Nice To Haves

Domain knowledge in Speech-to-Text: ASR architectures, audio processing, streaming inference.

Responsibilities

Raise the team's experimental velocity — make it faster to launch an experiment job, get a number back you can trust, and know what to try next.
Maintain and evolve our JAX training framework, keeping it scalable and efficient for large-scale distributed training runs on TPU.
Improve the data our models learn from: investigating quality issues, building the tooling to surface them, and turning what you find into measurable accuracy gains.
Analyze the accuracy of production models, build evaluation harnesses, and work out which improvements will matter most to customers.
Translate research prototypes into production-ready systems, refactoring and modernizing model architectures and infrastructure along the way.
Optimize production inference for speech language models, both from a serving architecture perspective and through advanced techniques such as quantization and speculative decoding.
Investigate and resolve performance bottlenecks across the stack, from low-level kernels (XLA, Pallas) to high-level system design.
Partner with researchers, infrastructure, and production engineering to trace problems to their real source and ship fixes that hold.