Software Engineer II

Microsoft • Redmond, WA

3d • $102,100 - $219,200

About The Position

The AI Infrastructure team builds and operates large-scale, highly reliable, and efficient GPU infrastructure powering Microsoft’s AI ecosystem. Its platforms support training and inference for Microsoft products and Azure AI Foundry services, and enable large-scale AI workloads across Azure and Microsoft partner clouds. As a Software Engineer, you will develop infrastructure and tools for model deployments, pre-training, post-training, and fine-tuning on NVIDIA and AMD GPUs, including systems running on large AI supercomputers. The role involves collaborating across engineering, research, and partner teams to improve reliability and performance, and contributing to the operation and continuous improvement of engineering systems and practices.

Requirements

• Bachelor's degree in Computer Science or a related technical field and 2+ years of technical engineering experience coding in languages such as C, C++, C#, Java, JavaScript, or Python, or equivalent experience.

Nice To Haves

• 2+ years designing, developing, and shipping high-quality software.
• 2+ years of experience with distributed systems and cloud-based infrastructure.
• 1+ year of experience with DevOps practices such as CI/CD, automated testing, and deployment.
• 2+ years of software development experience in C#, C++, Python, or similar languages.
• 2+ years of experience with containerization tools such as Docker and Kubernetes.
• Knowledge and hands-on experience with production ML systems, large-scale training infrastructure, NCCL, CUDA libraries, and tools.

Responsibilities

• Design, develop, and maintain AI infrastructure services in Go, Rust, Python, C++, and C#, deployed on large-scale Kubernetes clusters for inference, pre-training, and post-training workloads.
• Collaborate with engineers, researchers, and external partners to troubleshoot issues, improve reliability, and optimize large-scale AI training and inference systems.
• Build and enhance distributed systems that provide high reliability, low latency, operational efficiency, and strong security across Azure and partner cloud environments.
• Develop automation and tooling to improve GPU capacity utilization, streamline fleet operations, and enable efficient scaling of AI infrastructure.
• Provide operational support, technical leadership, and vision, and contribute to deployment, monitoring, and continuous improvement of engineering systems and practices.

Benefits

• The posting indicates that certain roles may be eligible for benefits and other compensation, with additional benefits and pay information available through the linked Microsoft careers page.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level