Engineering Manager, Kubernetes Bare Metal Infrastructure

Together AI•San Francisco, CA

6h•$300,000 - $350,000

About The Position

Together AI is hiring an Engineering Manager/Tech Lead Manager to lead the team responsible for running Kubernetes on bare-metal GPU infrastructure. The role owns how thousands of GPUs across the company’s data centers are provisioned, orchestrated, and kept healthy for customers training, fine-tuning, and serving models. The team operates the foundational bare-metal Kubernetes layer on which training runs, inference endpoints, and customer clusters depend. The manager sets technical direction for Kubernetes across multiple data centers and regions, drives zero-touch GPU-node provisioning, and owns cluster lifecycle at scale. The role also establishes reliability and efficiency goals, improves observability, and reduces the time from racked hardware to customer-ready capacity. It includes hiring and developing infrastructure and platform engineers, managing planning and delivery, establishing sustainable on-call practices, coordinating with operational and product partners, and representing platform priorities to leadership.

Requirements

8+ years in infrastructure, SRE, or platform engineering, including 2+ years managing engineers.
Deep hands-on experience running Kubernetes in production at scale, ideally on bare metal rather than only managed services.
Solid grounding in Linux systems, networking, and hardware, including kernels, drivers, BIOS/BMC, PXE boot, and out-of-band management.
Experience building fleet provisioning and lifecycle automation with tools such as Cluster API, Metal3, Ironic, MAAS, Ansible, or Terraform.
A track record of owning production systems with on-call, SLOs, and incident response.
Strong technical judgment, including the ability to review designs, debug alongside the team, and make clear trade-offs.
Clear communication with engineers, leadership, and external partners.

Nice To Haves

Experience operating large NVIDIA GPU clusters (H100, H200, B200, GB200) and using the NVIDIA GPU and Network Operators, DCGM, and NCCL.
Familiarity with InfiniBand, RoCE, SR-IOV, and high-performance storage for AI workloads.
Experience with Slurm, Kueue, Volcano, or other batch schedulers for training workloads.
Contributions to Kubernetes or related CNCF projects.
Experience at an AI lab, GPU cloud, or hyperscaler.

Responsibilities

Set the technical direction for running Kubernetes on bare metal across multiple data centers and regions.
Drive automated, zero-touch provisioning of GPU nodes, from firmware and OS imaging through cluster join and validation.
Own cluster lifecycle at scale, including upgrades, capacity expansion, node health checks, automated remediation, and decommissioning.
Make GPU scheduling, topology-aware placement, and high-performance networking—including InfiniBand, RoCE, and RDMA—first-class platform capabilities.
Define and meet SLOs for cluster availability, node bring-up time, and GPU utilization.
Invest in observability across hardware, networking, Kubernetes, and workloads so failures are found before customers are affected.
Reduce the time from racked hardware to customer-ready capacity.
Hire, manage, and develop infrastructure and platform engineers, setting a high bar for technical depth and operational ownership.
Run team planning, prioritization, and delivery, balancing roadmap work with fleet growth and production support.
Build a healthy on-call culture with sustainable rotations, blameless postmortems, and follow-through on remediation.
Work with data center operations, networking, hardware vendors, inference, training, and product teams to plan capacity and launch new hardware generations.
Represent platform needs and trade-offs to leadership and turn customer requirements into a clear roadmap.