About The Position
The Artificial Intelligence Performance team at Microsoft develops AI software that enables AI models to run across platforms, from AI supercomputers and servers to desktops, mobile devices, IoT devices, and internet browsers. The team works with internal and external hardware partners at the intersection of AI algorithms, purpose-built AI hardware, systems, and software. It owns inference performance for OpenAI and other state-of-the-art LLMs, collaborating directly with OpenAI on models hosted by Azure OpenAI that serve large workloads across Microsoft products. This hands-on technical role works across the AI software stack, including abstractions, programming models, compilers, runtimes, libraries, and APIs for large-scale model training and inference. The engineer benchmarks LLMs on GPUs and Microsoft hardware, debugs and optimizes performance, monitors models, and helps enable rapid, efficient deployment. The role requires software design and development skills, a track record of solving technical problems, and initiative in tackling challenges across the end-to-end AI stack.
Requirements
- Bachelor's degree in Computer Science or a related technical field and 4+ years of technical engineering experience coding in languages such as C, C++, C#, Java, JavaScript, or Python, or equivalent experience.
- Ability to meet Microsoft, customer, and/or government security screening requirements, including passing the Microsoft Cloud Background Check upon hire or transfer and every two years thereafter.
Nice To Haves
- Master's degree in Computer Science or a related technical field and 8+ years of technical engineering experience coding in languages such as C, C++, C#, Java, JavaScript, or Python; or a bachelor's degree in a related field and 12+ years of such experience; or equivalent experience.
- Technical background and a solid foundation in software engineering principles, computer architecture, GPU architecture, and hardware neural-net acceleration.
- Experience in end-to-end performance analysis and optimization of state-of-the-art LLMs, including proficiency with GPU profiling tools.
- Experience in DNN/LLM inference and one or more deep-learning frameworks such as PyTorch, TensorFlow, or ONNX Runtime, and familiarity with CUDA, ROCm, or Triton.
- Cross-team collaboration skills and a desire to collaborate with researchers and developers.
Responsibilities
- Identify and drive improvements to end-to-end inference performance of OpenAI and other state-of-the-art LLMs.
- Measure and benchmark performance on Nvidia and AMD GPUs and first-party Microsoft silicon.
- Optimize and monitor LLM performance and build software tooling that reveals performance opportunities across models, systems, and silicon, improving customer experience and reducing computing-fleet footprint.
- Build software tools that accelerate porting models to new Nvidia and AMD GPUs and enable rapid deployment at scale.
- Design, implement, and test functions or components for AI, DNN, and LLM frameworks and tools.
- Speed up or reduce the complexity of key components and pipelines to improve system performance or efficiency.
- Communicate and collaborate with internal and external partners.
- Embody Microsoft's culture and values.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
