AI Reliability Engineer (HPC)

Microsoft AI • Mountain View, NY

1d • $119,800 - $304,200 • Hybrid

About The Position

Microsoft AI is seeking an experienced Member of Technical Staff – AI Reliability Engineer to join its High Performance Computing (HPC) infrastructure team. The role combines software engineering and systems engineering to keep large-scale distributed AI infrastructure reliable and efficient, with very high uptime. The systems support MAI model training and inference.

MAI employees are expected to work from a designated Microsoft office at least four days a week if they live within 50 miles of that location in the U.S. or 25 miles outside the U.S., subject to local law and jurisdictional variation.

Requirements

• Bachelor’s degree in Computer Science or a related technical discipline and 4+ years of technical experience in Site Reliability Engineering, DevOps, or Infrastructure Engineering, or equivalent experience.

Nice To Haves

• Master’s degree in Computer Science or a related technical discipline and 2+ years of technical experience in Site Reliability Engineering, DevOps, or Infrastructure Engineering, or equivalent experience.
• Experience with Kubernetes, Docker, container orchestration, and CI/CD pipelines for ML training or inference workloads.
• Experience with public cloud platforms such as Azure, AWS, or GCP, including infrastructure-as-code.
• Experience with monitoring and observability tools such as Grafana, Datadog, or OpenTelemetry.
• Programming or scripting experience in Python, Go, or Bash.
• Experience with distributed systems, networking, storage, and high-performance computing (HPC).
• Experience operating GPU clusters and workload schedulers for ML/AI workloads.
• Experience with ML training or inference pipelines.
• Experience with capacity planning and cost optimization for GPU-based infrastructure.

Responsibilities

• Ensure uptime, resiliency, and fault tolerance of HPC clusters that power MAI model training and inference.
• Design and maintain monitoring, alerting, and logging systems that provide real-time visibility into HPC systems, including GPUs, clusters, storage, and networking.
• Build automation for deployments, incident response, scaling, and failover in CPU and GPU environments.
• Lead on-call rotations, troubleshoot production issues, conduct blameless postmortems, and drive continuous improvements.
• Ensure data privacy, compliance, and secure operations across model training and serving environments.
• Partner with ML engineers and platform teams to improve developer experience and accelerate research-to-production workflows.
• Embody Microsoft’s culture and values.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level