Senior Software Engineer - Performance Tooling

Microsoft • Redmond, WA

8d • $119,800 - $261,000

About The Position

The Artificial Intelligence (AI) Frameworks team at Microsoft develops AI software that enables models to run across systems ranging from AI supercomputers and servers to desktops, mobile devices, IoT devices, and internet browsers. The team works with internal and external hardware partners at the intersection of AI algorithms, purpose-built hardware, systems, and software. It owns inference performance for OpenAI and other state-of-the-art large language models, collaborating directly with OpenAI on models hosted by Azure OpenAI Service and used in major Microsoft products.

As a Senior Software Engineer - Performance Tooling, you will work across layers of the AI software stack, including abstractions, programming models, compilers, runtimes, libraries, and APIs, to support large-scale model training and inference. You will benchmark, debug, profile, monitor, and optimize model performance on GPUs and Microsoft hardware, helping models reach deployment with less time and hardware. The role involves collaborating with researchers and engineers to deliver scalable, production-ready performance improvements.

Requirements

• Bachelor's degree in Computer Science or a related technical field and 4+ years of technical engineering experience coding in languages including C++ or Python, or equivalent experience.
• Ability to meet Microsoft, customer, and/or government security screening requirements, including passing the Microsoft Cloud background check upon hire or transfer and every two years thereafter.
• 4+ years of practical experience working on high-performance applications and performance debugging and optimization on CPUs and GPUs.
• Experience in deep neural network (DNN) or large language model (LLM) inference and experience with one or more deep-learning frameworks such as PyTorch, TensorFlow, or ONNX Runtime; familiarity with CUDA, ROCm, and Triton.
• Technical background and a solid foundation in software engineering principles, computer architecture, GPU architecture, and hardware neural-network acceleration.
• Experience in end-to-end performance analysis and optimization of state-of-the-art LLMs and high-performance computing (HPC) applications, including proficiency using GPU profiling tools.
• Cross-team collaboration skills and willingness to collaborate with a team of researchers and developers.
• Ability to independently lead projects.

Nice To Haves

• Master's degree in Computer Science or a related technical field and 6+ years of technical engineering experience coding in languages including C++ or Python, or equivalent experience.
• Bachelor's degree in Computer Science or a related technical field and 8+ years of technical engineering experience coding in languages including C++ or Python, or equivalent experience.

Responsibilities

• Work across multiple layers of the AI software stack—including abstractions, programming models, compilers, runtimes, libraries, and APIs—to enable large-scale model training and inference.
• Benchmark OpenAI and other large language models for performance on GPUs and Microsoft hardware.
• Debug, profile, and optimize performance for training and inference workloads on CPUs and GPUs.
• Monitor performance regressions and drive continuous improvements to reduce time to deployment and hardware footprint.
• Collaborate across teams of researchers and engineers to deliver scalable, production-ready AI performance improvements.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior