Principal Software Engineer - AI Frameworks

Microsoft • ,

8d • $165,600 - $331,200

About The Position

The Microsoft AI Frameworks team develops software and performance systems that enable state-of-the-art AI models to run reliably and efficiently at cloud scale. The team owns the serving stack for OpenAI models and builds optimizations across model architectures, frameworks, compilers, runtimes, libraries, observability, benchmarking, and hardware platforms, including NVIDIA and AMD GPUs and Microsoft silicon. Engineers collaborate with model developers, researchers, hardware teams, and production services to accelerate model onboarding, improve performance and reliability, reduce deployment time and hardware footprint, and make performance insights into durable platform capabilities.

This is a hands-on technical leadership role focused on high-impact initiatives spanning AI frameworks and the serving stack for frontier models, including OpenAI GPT models. The Principal Software Engineer establishes technical direction, makes architectural decisions, and leads engineers across teams to design and deliver optimizations. The work aims to improve model velocity, platform efficiency, and production reliability. The role also identifies systemic opportunities, aligns stakeholders around durable architectures and measures of success, and contributes directly to critical implementation work. The role is part of Microsoft’s mission to empower people and organizations, with an emphasis on responsible innovation, shared progress, and meaningful impact.

Requirements

• Bachelor’s degree in Computer Science or a related technical field and 8+ years of technical engineering experience coding in languages such as C++ or Python, or equivalent experience.
• Ability to meet Microsoft, customer, and/or government security screening requirements, including passing the Microsoft Cloud Background Check upon hire or transfer and every two years thereafter.

Nice To Haves

• Master’s degree in Computer Science or a related technical field and 12+ years of technical engineering experience coding in languages including C++ or Python; or a bachelor’s degree in a related field and 15+ years of such experience; or equivalent experience.
• Experience serving as a technical lead for complex cross-team initiatives, including setting strategy, making architectural decisions, aligning stakeholders, and driving execution from concept through production.
• Extensive experience designing and shipping complex, high-performance, or distributed software systems.
• Deep foundation in software architecture, computer architecture, and accelerator-aware optimization.
• Advanced experience with LLM inference frameworks or AI accelerator kernel optimizations.
• Track record of creating reusable platforms, influencing senior technical and business stakeholders, developing other technical leaders, and scaling engineering practices across teams.

Responsibilities

• Own the technical vision, architecture, roadmap, and multi-release execution strategy for critical AI framework, performance, benchmarking, or developer-productivity capabilities.
• Lead technical alignment across teams by shaping architecture and priorities, resolving competing requirements, and ensuring researchers, product groups, infrastructure owners, and hardware partners commit to clear decisions, execution plans, and measurable outcomes.
• Mentor engineers, develop emerging technical leads, establish high standards for quality and maintainability, and foster an inclusive engineering culture that enables teams to execute at scale.
• Lead ambiguous cross-stack investigations and investments spanning models, frameworks, compilers, runtimes, systems, services, and silicon; set priorities, resolve trade-offs, and drive decisions.
• Set the engineering approach, delegate and unblock critical work, and contribute directly through prototypes, critical-path implementation, design and code reviews, complex debugging, and operational readiness.
• Develop common measurement, automation, observability, and engineering mechanisms that turn one-off analyses into scalable platform capabilities.
• Drive measurable improvements in model onboarding velocity, runtime performance, reliability, hardware utilization, and Azure capacity efficiency.
• Embody Microsoft’s culture and values.

What This Job Offers

• Job Type: Full-time
• Career Level: Principal