Cloud Principal Hardware Engineer

Microsoft•Redmond, WA
•$142,800 - $304,200

About The Position

The Cloud Principal Hardware Engineer will work within Microsoft Silicon, Cloud Hardware, and Infrastructure Engineering (SCHIE), specifically the Azure Memory and Storage Center of Excellence (AMS CoE), to scale Azure Fault Prediction and Detection (AFPD) across hardware components in the Azure fleet and lead hands-on hardware resiliency for the platform. AFPD uses hardware, firmware, and OS telemetry to detect and predict failures, move workloads away from failing nodes, and initiate targeted repairs. Repair results and supplier failure analysis inform improvements to the platform’s detection rules and models. The role covers server components including storage, memory, power supplies, CPUs, GPUs, NICs, HBAs/FPGAs, cables, and other server hardware. The engineer will set hardware direction for a platform operating at Azure scale, work with Azure engineering teams, component teams, and hardware suppliers, and improve fleet availability and customer uptime. The position also provides senior hardware escalation for live-site incidents and helps ensure new server generations and AI/GPU platforms have detection, isolation, and repair coverage before production launch.

Requirements

  • Bachelor's degree in Computer Science or a related technical field and 6+ years of technical engineering experience coding in languages including C, C++, C#, Java, JavaScript, or Python, or equivalent experience.
  • Ability to meet Microsoft, customer, and/or government security screening requirements, including the Microsoft Cloud Background Check upon hire or transfer and every two years thereafter.

Nice To Haves

  • Master's degree in Electrical Engineering, Computer Engineering, Mechanical Engineering, or a related field and 7+ years of technical engineering experience; or bachelor's degree in one of these fields and 8+ years of technical engineering experience; or equivalent experience.
  • 6+ years of hands-on failure analysis, debugging, or reliability engineering for server or datacenter hardware, covering two or more server component types.
  • 3+ years leading root-cause analysis and corrective actions with hardware suppliers, OEMs, or ODMs.
  • 2+ years using AI or machine learning in hardware engineering work, such as failure analysis, telemetry analysis, or automation.
  • 10+ years in server hardware reliability, failure analysis, or fleet resiliency at cloud or datacenter scale.
  • Technical depth in failure modes of at least one server component type, with working knowledge across the rest of the server, including BMC, PCIe, and firmware.
  • Experience turning failure signatures into automated detection rules, fault codes, or repair procedures for a large fleet.
  • Experience with predictive maintenance or anomaly detection on hardware telemetry, including setting precision targets and automated repair actions.
  • Proficiency with Kusto (Azure Data Explorer) or SQL, and Python or PowerShell, for large-scale telemetry analysis.
  • Experience with NPI and hardware qualification for server platforms.

Responsibilities

  • Lead failure analysis across server components by characterizing failure modes from fleet telemetry and logs and confirming root causes through repair results and supplier failure analysis.
  • Define signals, thresholds, targeted diagnostics, and repair recipes, and work with component teams on root cause analysis.
  • Improve failure isolation accuracy and the efficacy of detection and repair services so repairs address the source of failures.
  • Partner with software engineers and data scientists on detection rules and prediction models; set precision targets and validation gates before rules or models act on production nodes.
  • Use AI agents and machine learning to automate failure analysis, live-site triage, and supplier root-cause-analysis follow-up, and shorten the time from a new failure signature to a detection rule.
  • Lead root-cause analysis and corrective actions with server component teams, including firmware fixes, and feed learnings back into the platform.
  • Serve as the senior hardware escalation point for live-site incidents, from triage through root cause to fleet-wide mitigation.
  • Prepare new server generations and AI/GPU platforms for production with detection, isolation, and repair coverage at launch.
  • Track and improve metrics and mentor engineers across the team.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service