Senior Software Engineer - AI Infrastructure

Microsoft • Redmond, WA

1d

About The Position

Microsoft is seeking a Senior Software Engineer for its Azure AI infrastructure team. The team builds and operates a next-generation traffic shaping, scheduling, and optimization platform: a foundational control-plane service responsible for intelligent routing, capacity allocation, quota enforcement, and workload optimization for OpenAI models and other large-scale AI workloads across Azure. The role helps define and build systems that route and serve AI traffic globally, including the Traffic Shaper platform, which dynamically balances demand across AI endpoints, optimizes use of AI accelerators, and supports reliable, low-latency services for Microsoft Copilot, Azure OpenAI Service, and enterprise customers.

The engineer will design, develop, and operate large-scale distributed systems for routing, workload placement, capacity management, traffic shaping, prioritization, load balancing, quota management, fairness, fleet-wide optimization, telemetry, observability, operational controls, resiliency, failover, and incident mitigation. As a senior engineer, this person will provide technical leadership, drive architecture decisions, influence platform strategy, and work with teams across Azure AI, OpenAI, Infrastructure, Networking, and Core AI Services. The systems built will affect capacity allocation, traffic routing, and the efficiency of large AI workloads used by customers worldwide.

Requirements

• Bachelor's degree in Computer Science or a related technical field and 4+ years of technical engineering experience coding in languages such as C, C++, C#, Java, JavaScript, or Python, or equivalent experience.
• Ability to meet Microsoft, customer, and/or government security screening requirements, including passing the Microsoft Cloud Background Check upon hire or transfer and every two years thereafter.

Nice To Haves

• Master's degree in Computer Science or a related technical field and 6+ years of technical engineering experience coding in languages such as C, C++, C#, Java, JavaScript, or Python; or a bachelor's degree in Computer Science or a related technical field and 8+ years of such experience; or equivalent experience.
• Experience with telemetry, metrics pipelines, or resource scheduling systems.
• Familiarity with cloud platforms such as Azure, AWS, or GCP and container orchestration such as Kubernetes or Service Fabric.
• Experience operating large-scale services with stringent reliability and availability requirements.
• Experience with distributed storage systems, telemetry platforms, and real-time data processing.
• Familiarity with AI/ML infrastructure, GPU fleets, inference serving systems, or large-scale model deployment platforms.
• Experience leading cross-team initiatives and driving projects from design through production deployment and operation.

Responsibilities

• Design and implement highly scalable, reliable, and performant distributed systems for Azure AI traffic management and scheduling.
• Drive architecture and technical direction for routing, capacity allocation, and workload optimization services.
• Develop algorithms and control-plane systems to distribute AI workloads across global infrastructure while meeting latency, availability, and cost objectives.
• Collaborate with partner teams to onboard models, regions, and customer offerings while maintaining operational excellence.
• Build observability and operational tooling to monitor service health, capacity utilization, traffic patterns, and customer impact in near-real time.
• Lead investigations into complex production issues and drive improvements to reliability, resiliency, and platform automation.
• Identify and execute opportunities to improve fleet efficiency, reduce operational overhead, and optimize utilization of premium AI infrastructure.
• Mentor engineers, review designs and code, and raise engineering standards across the organization.
• Drive engineering excellence through testing, automation, documentation, and operational best practices.
• Influence long-term platform strategy and contribute to roadmap planning for next-generation AI infrastructure.

Benefits

• Certain roles may be eligible for benefits and other compensation; further benefits and pay information is available through the referenced Microsoft careers page.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior