Senior Staff/Principal Engineer, Foundation Model Compute Infra

Apple•Cupertino, CA

About The Position

As a Senior Staff/Principal Engineer on Apple’s Foundation Model Compute Infrastructure team, you will architect, build, and operate large-scale infrastructure powering foundation model training, fine-tuning, evaluation, and inference. The role addresses distributed scheduling, accelerator utilization, cluster management, fault tolerance, and large-scale job orchestration. You will work with cloud and hardware partners to bring new TPU generations into production and build tools and services that help researchers launch, monitor, debug, and profile workloads without requiring deep infrastructure expertise. You will develop model inference and fine-tuning services and build secure, scalable sandbox infrastructure for AI agents, code execution, and reinforcement learning environments at massive scale. Working closely with foundation model researchers and engineers, you will improve performance, reliability, scalability, and developer productivity. This is a hands-on technical leadership role involving production coding, reviews and debugging, complex architectural decisions, and the use of modern AI-assisted development tools to accelerate execution and raise quality.

Requirements

8+ years of experience designing and building distributed systems or large-scale infrastructure, with demonstrated technical leadership.
Strong hands-on experience with Kubernetes, cloud infrastructure, and operating production systems at scale.
Proficiency in Python, Go, C++, or other programming languages used for infrastructure and distributed systems.
Experience with accelerator infrastructure and machine learning training or inference workloads.
Excellent written and verbal communication skills, including experience authoring technical design documents and driving collaboration across teams.

Nice To Haves

Hands-on experience operating TPU, GPU, or other large-scale accelerator fleets.
Experience scaling distributed training or inference workloads across thousands of accelerators.
Familiarity with JAX, XLA, and distributed machine learning frameworks, including performance analysis and profiling.
Experience developing LLM-based agents or AI-assisted developer and infrastructure operations tools.
Deep knowledge of distributed scheduling, resource allocation, multi-tenant quota management, or cluster autoscaling.

Responsibilities

Lead by example through hands-on coding, debugging, and solving challenging architecture and implementation problems.
Design and build distributed scheduling, quota, and capacity management systems to maximize accelerator utilization across workloads and teams.
Improve fault tolerance, reliability, and recovery for large-scale training and inference workloads spanning thousands of accelerators, including automated hardware failure detection and remediation.
Architect and operate Kubernetes-based infrastructure, including accelerator provisioning, node lifecycle management, cluster orchestration, and autoscaling.
Collaborate with cloud and hardware partners to qualify, integrate, and deploy new generations of TPU hardware.
Design, build, and operate secure, high-density sandbox infrastructure for untrusted, model-generated code and AI agent workloads, supporting reinforcement learning training, agentic evaluation, and tool use.
Build self-service developer tools, APIs, and observability systems for job submission, monitoring, debugging, and performance profiling.
Design and develop scalable services for foundation model fine-tuning and inference.
Partner with researchers, ML framework engineers, and security teams to translate evolving research requirements into robust production infrastructure.
Mentor engineers, drive technical design reviews, and establish engineering and operational best practices across the team.