Capacity & Efficiency Infrastructure
About The Position
Microsoft AI is seeking a Member of Technical Staff to improve the management and efficiency of its compute fleet. The role contributes to training infrastructure for frontier-scale AI models, working across engineering and research to scale training recipes, implement distributed-training approaches, and ensure the reliability and performance of thousands of GPUs across a supercomputing fleet. The position involves building software and mathematical models to measure capacity usage and developing tools and techniques to improve it. Profiling, benchmarking, debugging, and fine-grained optimization are central to the work. The role partners with ML researchers, infrastructure engineers, and hardware teams to deliver measurable fleet-wide efficiency improvements and optimize for current and next-generation accelerators. Microsoft AI’s mission is to build AI that amplifies human potential and empowers people around the world. Employees who live within the stated distance of a designated Microsoft office are expected to work from that office at least four days per week, subject to local law and jurisdiction-specific variation.
Requirements
- Bachelor’s degree in Computer Science or a related technical discipline and 6+ years of technical engineering experience coding in languages including C, C++, C#, Java, JavaScript, or Python, or equivalent experience.
- Deep understanding of GPU architecture fundamentals and deep learning/LLM architectures.
- Deep experience profiling and analyzing performance in large-scale distributed computing systems.
- Deep experience profiling and analyzing performance in ML models, especially generative AI models.
Nice To Haves
- Bachelor’s degree in Computer Science or a related technical field and 10+ years of technical engineering experience coding in C++ or Python; or a master’s degree in Computer Science or a related technical field and 8+ years of such experience; or equivalent experience.
- Experience with low-level GPU programming (CUDA, Triton, NCCL) and frameworks such as PyTorch or JAX.
- Experience leading technical projects and supporting architectural decisions with data.
- Experience building infrastructure for large-scale machine learning or generative AI workloads.
- Experience in networking (InfiniBand, NVLink), storage systems, or distributed-training parallelisms.
- A track record of contributing to high-performance computing or large-scale AI infrastructure projects.
Responsibilities
- Design, implement, test, and optimize distributed training infrastructure in Python and C++ for large-scale GPU clusters.
- Build and evolve telemetry systems that provide visibility into infrastructure and ML model performance, utilization, and cost metrics.
- Profile, benchmark, and debug performance bottlenecks across compute, memory, networking, and storage subsystems.
- Drive architectural improvements across ML services that deliver measurable efficiency improvements.
- Build and evolve tools that automatically provide insights and recommendations to improve fleet-wide efficiency.
- Optimize collective communication libraries such as NCCL for emerging NVLink and InfiniBand topologies.
- Partner with ML researchers and infrastructure engineers to understand future plans and needs, and develop plans to balance growth with efficiency.
- Collaborate with hardware teams to optimize for next-generation accelerators, including NVIDIA and MAIA.
- Embody Microsoft’s Culture and Values.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
