Principal Software Engineer, AI Infrastructure

Microsoft•Redmond, WA

About The Position

Microsoft’s AI Infrastructure team builds the platform that routes AI traffic across global infrastructure, helping services such as Azure OpenAI Service and Microsoft Copilot remain fast and reliable while making effective use of GPU capacity. The Principal Software Engineer will guide how AI requests are routed and prioritized and matched to available computing capacity around the world. The role involves shaping large-scale distributed systems, collaborating with partner teams across Microsoft and OpenAI, and supporting the growth of other engineers. The position offers the opportunity to influence a large AI platform and develop expertise in optimization, efficiency, and technical leadership. The engineer will own the design of systems that manage AI traffic and computing capacity across regions, develop and review code for routing, prioritization, and capacity planning, and partner with product and program managers to turn strategy into a dependable roadmap. The role also includes safe feature rollouts, on-call responsibility for critical services, incident reviews, monitoring improvements, launch readiness, security and privacy practices, mentoring, and helping the team responsibly use AI tools throughout the software development lifecycle.

Requirements

Bachelor’s degree in Computer Science or a related technical field and 6+ years of technical engineering experience building distributed systems, with coding in languages such as C#, Java, or Go, or equivalent experience.
Ability to meet Microsoft, customer, and/or government security screening requirements, including passing the Microsoft Cloud Background Check upon hire or transfer and every two years thereafter.

Nice To Haves

Master’s degree in Computer Science or a related technical field and 8+ years of technical engineering experience building distributed systems using languages such as C#, Java, or Go, or equivalent experience.
Bachelor’s degree in Computer Science or a related technical field and 12+ years of technical engineering experience building distributed systems using languages such as C#, Java, or Go, or equivalent experience.
5+ years of experience running distributed services on public cloud platforms such as Microsoft Azure, Amazon Web Services, or Google Cloud Platform.
3+ years of experience with systems that route or schedule traffic, or use optimization methods to allocate computing resources.
Experience with Python, Docker, and Kubernetes.
Experience with AI or machine learning infrastructure, such as serving models in production or managing GPU fleets.
Experience leading projects across three or more partner teams from design through launch and operation.
Experience being on call for highly available services and measurably reducing incidents.

Responsibilities

Own the overall design of how AI traffic and computing capacity are managed across regions.
Guide design reviews and help evaluate tradeoffs among speed, reliability, scale, and cost.
Write and review clear, secure, well-tested code for routing, prioritization, and capacity planning.
Establish coding practices that other teams can reuse.
Work with product and program managers to translate the long-term vision into a dependable, step-by-step roadmap.
Use feature flags and gradual rollouts to test changes safely, measure impact, and roll back quickly when needed.
Participate in the on-call rotation as a designated responsible individual for critical AI services.
Lead incident reviews and improve monitoring to detect issues earlier and reduce their frequency.
Partner with product, research, and infrastructure teams to understand customer needs as AI models and capabilities launch.
Coordinate shared dependencies and ensure testing and support paths are ready before launches.
Support security, privacy, and compliance through threat modeling, safe deployment checks, and clear records.
Mentor engineers and share knowledge about building reliable systems at scale.
Help the team use AI tools and agents across the software development lifecycle, including coding, testing, and investigating live issues.
Follow Responsible AI practices and track the impact of AI tools on quality and productivity.