Software Engineer, ML Infrastructure, Reality Labs Research

Meta•Redmond, WA

3d•$183,997 - $257,000

About The Position

Meta's mission is to give people the power to build community and bring the world closer together. Through our family of apps and services, we're building a different type of company that connects billions of people around the world, gives them ways to express what matters most to them, and helps bring people closer together. Whether we're creating new products or helping a small business expand its reach, people at Meta are builders at heart. Our global teams are constantly iterating, solving problems, and working to empower people around the world to build community and connect in meaningful ways. At Meta Reality Labs Research (RL-R), our goal is to explore, innovate, and design novel interfaces and hardware subsystems for the next generation of virtual, augmented, and mixed reality experiences. Our research team has built and operates a shared ML compute platform: a large-scale, Kubernetes-based multi-use cluster that powers model training, inference, rendering, and unique research workloads with interactive development for research and engineering teams across RL-R. We are looking for a senior software engineer to set technical direction for this platform, driving its reliability, scale, and efficiency as demand grows, and partnering across teams to turn evolving ML workload needs into platform capabilities. In this role, you will be a hands-on technical leader who has experience navigating ambiguous problem spaces and driving them to concrete solutions, operates complex distributed systems in production, and elevates team practices through code reviews, design feedback, and knowledge sharing. Meta is committed to providing reasonable accommodations for qualified individuals with disabilities and disabled veterans in our job application procedures. If you need assistance or an accommodation due to a disability, you may contact us at [email protected]

Requirements

Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
8+ years of professional software engineering experience, including technical leadership of complex, cross-team infrastructure efforts
Experience building and maintaining production systems in Python (and/or Go, C++, or a similar language)
Proven experience designing, operating, and scaling large distributed systems in production, including production reliability / oncall ownership
Hands-on experience running container orchestration (Kubernetes or comparable) at scale
Experience with one or more of: GPU/ML training infrastructure, cluster/batch scheduling, distributed storage, or high-performance networking
Demonstrated ability to drive ambiguous, organization-level technical problems to resolution and to set multi-quarter technical direction

Nice To Haves

Demonstrated ability to integrate AI tools to optimize/redesign workflows and drive measurable impact (e.g., efficiency gains, quality improvements)
Large-scale distributed or parallel storage systems (e.g., HDFS, Isilon, Lustre)
Deep Linux systems experience, performance tuning, and observability (metrics, tracing, alerting)
GPU cluster operations (NVIDIA DGX, H100/A100), NCCL, and high-performance networking (InfiniBand / RoCE)
Distributed ML training frameworks (e.g., PyTorch) and the surrounding training/inference stack
Demonstrated ongoing AI skill development (e.g., prompt/context engineering, agent orchestration) and staying current with emerging AI technologies
Kubernetes operators/CRDs and batch schedulers (e.g., Volcano, Slurm, Kueue)
Experience with Hack/PHP
Experience adhering to and implementing responsible, ethical AI practices (e.g., risk assessment, bias mitigation, quality and accuracy reviews)

Responsibilities

Set technical direction for and lead the design, build, and operation of large-scale, multi-tenant GPU/ML training, inference, rendering, and general compute infrastructure on Kubernetes
Drive cross-team and org-level initiatives that improve cluster reliability, scheduling efficiency, utilization, and scale, including datacenter and hardware migrations and expansions
Our reliability for the cluster end to end: participate in and improve oncall and incident response, and build the automation and observability that reduces operational costs
Define and drive best practices in system design, testing, code health, and operational excellence across the team
Partner with research and infrastructure teams to translate ML workload requirements into platform capabilities, and influence architecture and technical decisions beyond your immediate team
Mentor engineers, review designs, and improve the overall technical quality of the team's work through code reviews, design guidance, and knowledge sharing