Technical Program Manager, Inference

Anthropic•San Francisco, NY

5d•Hybrid

About The Position

Anthropic is seeking a Technical Program Manager for its Inference organization, which serves Claude to millions of users and enterprise customers through Anthropic products and major cloud platforms. The role drives programs that scale inference across accelerator and cloud platforms, spanning serving-fleet infrastructure, engineering velocity, the inference engine, and performance. The technical environment includes accelerator kernels, an in-house inference engine, distributed routing, autoscaling, and capacity management across GPUs, TPUs, and Trainium. This is an outcome-owning role rather than a status-tracking position. The TPM will lead complex, cross-functional efforts that may span Research, Safeguards, Security, Compute, Infrastructure, API, and cloud and silicon partners. They will build only the program mechanisms needed to move ambiguous work forward and retire mechanisms that no longer help. The role also includes exploring how Claude can support AI-assisted program management and fleet operations. The position works closely with production, research, and safeguards stakeholders and owns inference readiness for model launches alongside launch TPMs. Anthropic currently expects staff to work from one of its offices at least 25% of the time, though some roles may require more office time.

Requirements

At least 7 years of experience as a TPM or in an equivalent engineering or SRE lead role, owning large, multi-team infrastructure programs end to end in production distributed systems, ML infrastructure, or cloud platforms.
Ability to discuss program outcomes in detail, including delays and how they were addressed.
Technical fluency in large-scale serving infrastructure, including request routing and load balancing, autoscaling, capacity planning, cluster orchestration, deployment and rollout, and observability.
Ability to challenge technical designs, identify risks, and contribute beyond tracking.
Working understanding of LLM inference hardware use—including prefill and decode, batching, caching, and compute, memory-bandwidth, and interconnect trade-offs—or a record of rapidly learning unfamiliar technical domains.
Track record of creating program mechanisms from scratch for ambiguous, fast-moving efforts and retiring processes that are no longer useful.
Ability to drive outcomes across organizations the role does not manage, despite competing priorities and unclear decision ownership.
Strong interpersonal judgment, including knowing when to push and when to listen, and ability to earn the trust of senior engineers and researchers.
Strong written communication, including plans, reviews, and executive updates used by multiple teams.
Low ego and high ownership, with willingness to do whatever the program needs, such as writing scripts, running incident bridges, or pursuing quotas.
Care for Anthropic’s mission and for serving frontier AI safely and reliably.
Bachelor’s degree or an equivalent combination of education, training, and/or experience.
Relevant field of study, demonstrated through coursework, training, or professional experience.

Nice To Haves

Direct experience with LLM or ML inference serving at scale, such as model servers, batching and scheduling, KV caching, quantization, speculative decoding, or multi-model routing.
Experience with GPU, TPU, or Trainium infrastructure, including cluster bring-up, hardware qualification, or running a workload across multiple accelerator platforms.
Experience managing capacity across multiple clouds or regions, including demand forecasting, headroom policy, cost modeling, or compute-service unit economics.
Experience with developer productivity, release engineering, or CI/CD at scale, especially for hardware-dependent workloads, and using engineering metrics to measurably improve velocity.
Experience running performance or efficiency programs, with knowledge of rooflines, kernels, ML compilers such as XLA, Triton, or Neuron, numerics, or collective communication sufficient to reason about trade-offs with engineers.
Familiarity with Kubernetes-based orchestration and job scheduling.
Experience working with research teams and translating their needs into infrastructure requirements.
Practical experience using AI tools in personal workflows, including building automations or agents, and clear ideas about where those tools are and are not helpful.

Responsibilities

Own programs to scale Claude’s serving fleet across accelerator and cloud platforms, from early direction through delivery.
Drive new capacity online end to end across compute, infrastructure, security, networking, and inference teams.
Build visibility into supply, demand, and utilization with capacity and finance partners, and drive clear trade-off decisions among utilization, cost, latency, and reliability.
Lead reliability programs and large migrations without disrupting production traffic or research, and ensure incident follow-ups are incorporated into team roadmaps.
Improve inference-engineering productivity by defining relevant metrics, identifying bottlenecks, and improving software development, testing, and release processes across platforms in coordination with Anthropic’s central Infrastructure organization.
Coordinate roadmaps and priorities across production, research, and safeguards needs that depend on the same teams.
Track performance and efficiency work from opportunity through measured production impact, treating correctness as a release criterion.
Own inference readiness for model launches alongside launch TPMs.
Build appropriate program mechanisms for ambiguous, fast-moving work and retire processes that no longer provide value.
Explore and advance the use of Claude and other AI tools in program management and fleet operations.