Software Engineer II

Microsoft•Redmond, WA

About The Position

The AI Infrastructure team builds and operates large-scale, reliable, and efficient GPU infrastructure powering Microsoft’s AI ecosystem. Its platforms support training and inference for Microsoft products and Azure AI Foundry services, including workloads for OpenAI and open-source models. The infrastructure supports large-scale AI deployments, pre-training, post-training, and fine-tuning using current-generation NVIDIA and AMD GPUs in Azure and Microsoft partner clouds, including large AI supercomputers. As a Software Engineer on the team, you will develop infrastructure services and tools for AI workloads, collaborate to improve system reliability and performance, and help operate and scale distributed systems across cloud environments. Microsoft describes its mission as empowering every person and organization to achieve more and emphasizes growth mindset, innovation, collaboration, respect, integrity, accountability, and inclusion.

Requirements

Bachelor’s degree in Computer Science or a related technical field and 2+ years of technical engineering experience coding in languages including C, C++, C#, Java, JavaScript, or Python, or equivalent experience.

Nice To Haves

2+ years designing, developing, and shipping high-quality software.
2+ years of experience with distributed systems and cloud-based infrastructure.
1+ year of experience with DevOps practices, including CI/CD, automated testing, and deployment.
2+ years of software development experience in C#, C++, Python, or similar languages.
2+ years of experience with containerization tools such as Docker and Kubernetes.
Knowledge and hands-on experience with production ML systems, large-scale training infrastructure, NCCL, CUDA libraries, and tools.

Responsibilities

Design, develop, and maintain AI infrastructure services in Go, Rust, Python, C++, and C#, deployed on large-scale Kubernetes clusters to support inference, pre-training, and post-training workloads for state-of-the-art AI models.
Collaborate with engineers, researchers, and external partners to troubleshoot issues, improve reliability, and optimize performance of large-scale AI training and inference systems.
Build and enhance distributed systems that provide high reliability, low latency, operational efficiency, and strong security across Azure and partner cloud environments.
Develop automation and tooling to improve GPU capacity utilization, streamline fleet operations, and enable efficient scaling of AI infrastructure.
Provide operational support, technical leadership, and vision while contributing to deployment, monitoring, and continuous improvement of engineering systems and practices.