Principal Site Reliability Engineer

Cerebras Systems•Sunnyvale, CA

3d•Hybrid

About The Position

Cerebras Systems is building a high-performance SRE function to support one of the world’s fastest-growing AI inference services, powered by the Wafer-Scale Engine (WSE). This team will help deliver world-class, ultra-reliable inference infrastructure for leading model builders such as OpenAI and other frontier labs. As a Principal SRE, you will define and drive the technical architecture for scaling our inference fleet through self-service delivery, shared observability, capacity orchestration, rollout safety, and operational automation. This role starts with 2–3 weeks of hands-on operational immersion to build deep context on the current stack, production pain points, and high-stakes workflows. From there, your mandate shifts to architecting the “tomorrow” layer: a unified capacity management and production control plane that enables reliable capacity planning, workload placement, rollout safety, validation, and operational decision-making across large-scale inference infrastructure. Success in the first year means core engineering teams, product managers, external customers, and cluster stakeholders can execute critical operational workflows through self-service systems with strong guardrails, clear ownership, and minimal dependency on expert SRE operators. You will collaborate with the tech leads and the leadership team across core, cluster, cloud, and product stakeholders. This work will shift reliability from an ops-only burden to a shared engineering discipline that underpins frontier AI inference at scale. If you are a proven Principal engineer who enjoys turning complexity into elegant reliability at scale, this is your chance to lead this transformation from the front. This role does not require 24/7 on-call rotations.

Requirements

15+ years in SRE, infrastructure engineering, or platform engineering, with a record of setting technical direction and delivering reliability improvements at large scale in FAANG, hyperscaler, frontier AI, or similarly demanding production environments.
Deep experience with large-scale compute fleets, internal control planes, schedulers, orchestration systems, capacity management, and reliability automation.
Experience defining and driving cross-team architecture for production control planes, capacity orchestration, fleet management, or self-service infrastructure platforms with clear operational ownership.
Strong judgment in converging fragmented workflows, tools, and teams into coherent architectures that improve reliability, efficiency, and operational leverage.
Ability to lead complex, ambiguous technical programs end to end; influence senior cross-functional stakeholders; mentor senior engineers; and communicate technical strategy clearly.
Hands-on experience with production observability, incident response, and SLO-based reliability management across metrics, logs, traces, alerting, dashboards, and operational review loops.

Nice To Haves

Experience with Bazel or other large-scale build systems in production.
Background in AI/ML inference systems, including model serving runtimes, disaggregated inference, GPU orchestration, latency and accuracy SLOs, or drift monitoring.
Prior work on predictive autoscaling, chaos engineering, or cost-aware capacity management for compute-intensive workloads.

Responsibilities

Define and implement a robust strategy for delivering and running software reliably and at scale across multiple datacenters and cloud-based solutions.
Architect self-service platforms and internal tooling that let product teams, external customers, and cluster operators safely trigger and observe critical workflows with minimal handoffs.
Define and evolve reliability practices for inference workloads, including SLOs and SLIs for latency, throughput, and accuracy stability; error budgets; blameless postmortems; chaos testing; and capacity forecasting across multi-datacenter and on-prem environments.
Mentor senior SREs, support critical incident escalations, and use production pain points to prioritize the highest-leverage automation work.
Measure and drive impact through clear metrics, including toil reduction, deployment velocity, SLO compliance, MTTR, and adoption of self-service workflows.