Senior Software Engineer- AI Frameworks
About The Position
The Microsoft AI Frameworks team develops software, performance systems, and engineering tools that enable state-of-the-art AI models to run reliably and efficiently at cloud scale. The team works across model architectures, frameworks, compilers, runtimes, libraries, observability, benchmarking, and hardware platforms, including NVIDIA and AMD GPUs and Microsoft silicon. Senior Software Engineers partner with model developers, researchers, hardware teams, and production services to accelerate model onboarding, improve performance and reliability, reduce deployment time and hardware footprint, and turn performance insights into durable platform capabilities. This is a hands-on individual-contributor role focused on ambiguous, end-to-end systems problems. Engineers combine software engineering fundamentals with curiosity about AI workloads, disciplined measurement, and collaboration across organizational boundaries to deliver production impact. The role owns significant components and projects across AI performance, benchmarking, automation, and developer tooling, translating model and platform needs into robust software, investigating complex performance and reliability issues, and working with partner teams to deliver measurable improvements into production.
Requirements
- Bachelor’s degree in Computer Science or a related technical field and 4+ years of technical engineering experience coding in languages such as C++ or Python, or equivalent experience.
- Ability to meet Microsoft, customer, and/or government security screening requirements, including passing the Microsoft Cloud Background Check upon hire or transfer and every two years thereafter.
Nice To Haves
- Expertise in GPU or equivalent accelerator programming and kernel optimization using CUDA, Triton, or comparable technologies.
- Experience using profiling and benchmarking to deliver measurable performance improvements, with an understanding of memory hierarchy, parallel execution, synchronization, and hardware utilization.
- Familiarity with vLLM, SGLang, or equivalent inference frameworks, including model execution, scheduling, batching, and KV-cache mechanisms.
- Demonstrated technical ownership and cross-team collaboration, including independently delivering complex engineering changes into production.
- Ability to use AI-assisted development tools effectively while maintaining sound engineering judgment and accountability for correctness, performance, and maintainability.
Responsibilities
- Design, implement, test, and operate production-quality components across AI frameworks, runtimes, benchmarking systems, performance tooling, and service integrations.
- Benchmark, profile, debug, and optimize large language model training and inference workloads across GPUs and Microsoft hardware.
- Build automation and observability that detect regressions, improve reproducibility, surface actionable insights, and accelerate model and hardware onboarding.
- Drive scoped projects from problem definition through deployment, balancing delivery speed, maintainability, reliability, and measurable customer or capacity impact.
- Partner with researchers, model teams, infrastructure owners, and hardware vendors to diagnose cross-stack issues and deliver production-ready solutions.
- Contribute to technical design reviews, engineering standards, operational health, and mentoring of other engineers.
- Embody Microsoft’s culture and values.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
