Principal Engineer Capacity/Compute

Anthropic•San Francisco, CA

4d•Hybrid

About The Position

Anthropic’s Capacity Engineering team ensures that the company’s large and rapidly growing infrastructure fleet is accounted for, well-utilized, and efficiently allocated across accelerator and CPU families and cloud providers. The team builds the data, tooling, and operational systems used to plan, measure, and maximize utilization across first-party and third-party compute. This hands-on role builds production systems including telemetry data pipelines, fleet observability tools, performance instrumentation, planning and allocation platforms, cost attribution, and forecasting systems. The work spans data engineering, systems engineering, and observability, and involves operating Kubernetes-native infrastructure at scale. The engineer will collaborate with research engineering, infrastructure, inference, and finance teams in a high-autonomy, ambiguous environment. This is a pipeline role across data platform, planning, efficiency, and attribution and forecasting; the primary area of focus depends on the candidate’s background and business priorities, and the boundaries between areas are fluid. The role includes building data products for consumers ranging from research engineers to finance and leadership, with responsibility for requirements, schema contracts, data discoverability, on-call, and SLOs. It also directly informs decisions about a major area of company spending. Anthropic is a public benefit corporation headquartered in San Francisco. The company describes its mission as creating reliable, interpretable, and steerable AI systems that are safe and beneficial. It offers competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and office space for collaboration. The location policy currently expects staff to work from an office at least 25% of the time, though some roles may require more.

Requirements

A strong track record building and operating production systems.
Production-quality Python and SQL skills; pipeline code is primarily Python, and the presentation layer uses BigQuery SQL, including table-valued functions and views.
Deep experience with at least one major cloud provider: Amazon Web Services, Google Cloud, or Microsoft Azure, including its operations.
Experience with observability tooling, including Prometheus, PromQL, and Grafana, and writing recording rules and building monitoring that engineering teams rely on.
Ability to gather requirements independently and work across organizational boundaries in an ambiguous environment with limited direction.
A bachelor’s degree or an equivalent combination of education, training, and/or experience.
Education or experience in a field relevant to the role, demonstrated through coursework, training, or professional experience.

Nice To Haves

Experience with capacity planning, resource management, or cost attribution systems at a hyperscaler or in a large-scale machine learning environment.
Product engineering or developer experience.
Experience with scheduling and packing efficiency, or profiling-driven optimization of large distributed workloads.
Multi-cloud data ingestion experience, especially normalizing billing exports, reservation APIs, on-demand capacity reservations, commitments, and vendor telemetry across providers with different billing arrangements.
Total cost of ownership and forecasting experience, including analyzing whether infrastructure growth is causal or correlated with business drivers.
Accelerator infrastructure familiarity, such as GPU metrics (DCGM), TPU utilization, Trainium power and utilization metrics, or hardware-level experience with machine learning training and inference systems.
Experience building internal data products with self-service access, schema contracts, API serving, documentation, and discoverability.
Experience with storage efficiency, retention, and lifecycle programs at exabyte scale.

Responsibilities

Build the planning and allocation stack used by leadership to allocate capacity, by teams to plan against allocations, and by the scheduler to enforce them, including cross-region and cross-provider placement, guardrails, queueing, and occupancy KPIs.
Drive efficiency programs such as stranding and rightsizing, unused-capacity recovery, and job-level utilization across training, inference, and evaluation systems.
Establish per-configuration efficiency baselines and work with system-owning teams to improve utilization.
Own attribution and forecasting by reconciling billing across more than ten providers against telemetry and internal systems, attributing spend to workloads, and turning demand signals and research roadmaps into a defensible compute plan and supply pipeline.
Build data pipelines that ingest occupancy, utilization, and cost data from a diversifying fleet into BigQuery, owning completeness, latency SLOs, and gap detection.
Integrate new providers into the data platform.
Operate Kubernetes-native systems at scale, including collection agents, workload labeling, and taint, reservation, and scheduling behavior that determines usable capacity.
Gather requirements, define schema contracts, and design data outputs as products for consumers ranging from research engineers to a CFO.
Provide on-call support and maintain SLOs for load-bearing data and operational surfaces.
Build production systems and write production-quality code as part of day-to-day work.