Senior Software Engineer, AI Infrastructure

Microsoft • Redmond, WA

8d • $119,800 - $261,000

About The Position

The Senior Software Engineer, AI Infrastructure will design, build, and operate services that route and prioritize AI requests and match them with available computing capacity across Microsoft’s global infrastructure. The role owns the design of key platform components, partners with other teams to deliver new AI models and capabilities, and helps maintain production service health. It offers the opportunity to deepen expertise in globally distributed systems and AI model serving while growing as a technical leader and mentor.

The team supports Azure OpenAI Service and Microsoft Copilot by directing AI traffic to appropriate GPUs and regions, aiming to keep services fast, reliable, fair, and dependable while using GPU resources effectively. The position includes participation in an on-call rotation and responsibility for building monitoring, improving troubleshooting, and preventing incidents. Microsoft describes its workplace culture as grounded in growth mindset, innovation, collaboration, respect, integrity, accountability, and inclusion.

Requirements

• Bachelor's degree in Computer Science or a related technical field and 4+ years of technical engineering experience building distributed systems using languages such as C#, Java, or Go, or equivalent experience.
• Ability to meet Microsoft, customer, and/or government security screening requirements, including passing the Microsoft Cloud Background Check upon hire or transfer and every two years thereafter.

Nice To Haves

• Master's degree in Computer Science or a related technical field and 6+ years of technical engineering experience building distributed systems using languages such as C#, Java, or Go; or a bachelor's degree in a related field and 8+ years of such experience; or equivalent experience.
• 3+ years of experience running distributed services on public cloud platforms such as Microsoft Azure, AWS, or GCP.
• 2+ years of experience with systems that route or schedule traffic, or allocate computing resources.
• Experience with Python, Docker, and Kubernetes.
• Experience with AI or machine learning infrastructure, such as serving models in production or managing GPU fleets.
• Experience delivering projects with partner teams from design through launch and operation.
• Experience being on call for highly available services and improving monitoring to reduce incidents.

Responsibilities

• Own the design of key platform components for routing AI traffic and managing computing capacity; write design proposals, compare options, and ensure designs meet speed, reliability, scale, and cost goals.
• Write clear, secure, well-tested code and provide thoughtful, timely code reviews.
• Break projects into dependable milestones with program managers; use feature flags and gradual rollouts to test changes, measure impact, and enable quick rollbacks.
• Participate in the on-call rotation as a designated responsible individual; add monitoring and dashboards, improve troubleshooting guides, and help reduce incidents.
• Work with product managers and partner teams to understand customer needs and translate them into technical requirements.
• Build security, privacy, and compliance into work from design through deployment; automate testing and releases and share learnings with the team.
• Use AI tools throughout the software development lifecycle, including coding, testing, and live-issue investigation; take responsibility for AI-generated work and help teammates do the same.
• Embody Microsoft's culture and values.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior