AI Inference Systems & Optimization

Microsoft AI•Redmond, WA

9d•$142,800 - $304,200•Hybrid

About The Position

Microsoft AI is seeking a Member of Technical Staff, Inference Systems Research to contribute to the development of next-generation AI infrastructure for frontier model development. The Inference Systems team is an applied research group responsible for building and maintaining the infrastructure that powers Microsoft’s frontier models. This team collaborates with researchers to ensure efficient operation of models across various applications, including reinforcement learning, synthetic data generation, evaluations, and production, utilizing one of the world's largest compute infrastructures. The team is fully integrated, managing all aspects from kernels and inference algorithms to model architecture co-design, ASIC co-design, distributed systems, and profiling/testing tools. A significant research focus is on optimizing the reinforcement learning loop. The team is embedded within Microsoft AI’s research organization to foster close collaboration with researchers.

Requirements

Bachelor's degree in Computer Science or related field, or equivalent experience.
6+ years of software engineering experience.
Experience building large-scale distributed systems.

Nice To Haves

Understand modern generative AI architectures and how to optimize them for inference.
Familiarity with the internals of open-source inference frameworks like vLLM and SGLang.
Value clear communication, improving team processes, and being a supportive team player.
Results-oriented, have a bias toward action, and enjoy owning problems end-to-end.
Have or can quickly gain familiarity with modern Python and its tooling, PyTorch, Nvidia GPU kernel programming and optimization, Infiniband, and NVLink.
Experience with AI/ML infrastructure, inference, or generative AI.
Experience optimizing large-scale systems for performance and efficiency.

Responsibilities

Work alongside researchers and engineers to implement frontier AI research ideas.
Introduce new systems, tools, and techniques to improve model inference performance.
Build tools to help debug performance bottlenecks, numeric instabilities, and distributed systems issues.
Build tools and establish processes to enhance the team’s collective productivity.
Find ways to overcome roadblocks and deliver your work to users quickly and iteratively.