Production Systems Engineer, Fleet AI Systems

Meta•Austin, TX

2d•$208,000 - $289,000

About The Position

Meta is seeking a Production Systems Engineer to lead the reliability, scalability, and operational excellence of its AI infrastructure fleet. The role supports the hardware and software stack powering AI training and inference workloads across tens of thousands of accelerators and servers. It drives strategy for fleet health, automated diagnostics, and lifecycle management of AI-optimized hardware, partnering with hardware design, silicon, software, and data center operations teams to ensure AI systems are production-ready and operate efficiently.

Requirements

Bachelor's degree in Computer Science, Computer Engineering, a relevant technical field, or equivalent practical experience.
12+ years of experience in production systems engineering, server hardware support, or infrastructure reliability engineering at hyperscale.
12+ years of experience troubleshooting and root-causing failures across the hardware and software stack, including CPUs, accelerators, memory, networking, storage, firmware, and operating systems.
Experience leading large-scale technical programs across hardware, firmware, and software engineering organizations with demonstrated organizational impact.
Experience designing and implementing automated diagnostics, test frameworks, or remediation systems for large server fleets.
Experience with Linux systems administration and scripting or programming in Python or equivalent languages for infrastructure tooling.

Nice To Haves

Experience supporting AI training or inference infrastructure, including GPU clusters, high-speed interconnects such as InfiniBand or RoCE, and AI accelerator hardware at fleet scale.
Experience engaging directly with silicon vendors or ODMs on firmware qualification, failure analysis, and hardware specification development for AI compute systems.
Experience in post-production hyperscale environments managing full hardware lifecycle progressions for AI or HPC server platforms.
Experience with predictive fleet health modeling, telemetry pipelines, or data visualization platforms applied to large-scale hardware operations.

Responsibilities

Define and drive long-term technical strategy for fleet health monitoring, automated remediation, and lifecycle management of AI compute infrastructure at hyperscale.
Lead cross-functional initiatives spanning hardware design, firmware, systems software, and data center operations to bring new AI server platforms from pre-production validation through mass deployment.
Architect and oversee large-scale test automation frameworks for AI accelerator systems, covering GPU, TPU, and custom silicon configurations across diverse fleet topologies.
Establish organizational standards for root cause analysis, failure mode classification, and escalation workflows for AI system incidents at fleet scale.
Engage vendors and partners on hardware specifications, firmware quality, and test methodology improvements to increase AI infrastructure reliability.
Develop data-driven visibility platforms that surface fleet health signals, predictive failure indicators, and capacity risk across AI training and inference clusters.
Lead investigations into systemic hardware and software failures affecting AI workload availability, and produce durable solutions and documented learnings.
Mentor and provide technical direction to engineers across the production systems organization, shaping engineering practices and team capability.
Collaborate with capacity engineering and AI platform teams to translate fleet reliability requirements into hardware procurement, qualification, and deployment criteria.
Influence internal tooling and automation roadmaps to reduce mean time to detection and resolution for AI fleet incidents.