Technical Program Manager, Cloud Inference

Modular•United States / Canada,
•$143,400 - $277,200•Hybrid

About The Position

The Technical Program Manager will support Cloud Inference, which builds and operates Kubernetes-native, cluster-level serving infrastructure for Modular's inference products. The infrastructure runs large language models in production across clouds and hardware vendors. The team owns cluster orchestration, multi-cluster routing, and the deployment path for safely releasing model and engine versions. This is a hands-on technical role between Cloud Inference and the MAX product and customer-facing teams. The program manager turns goals such as higher GPU throughput, lower cost per token, faster model bring-up, and fewer production incidents into shipped work. The role works with live customer-traffic systems, where rollout decisions directly affect latency and spending. The position partners with engineering and product leaders to structure planning, tracking, and communication; participates in design and capacity discussions; helps teams develop realistic plans; surfaces risks early; and shapes engineering decisions across the organization. Candidates may work from the Los Altos, California office or remotely from home, and new-hire onboarding takes place in person at the Los Altos office. Candidates based in or open to relocating to the United States or Canada are welcome.

Requirements

  • Bachelor's degree in a technical field or equivalent practical experience.
  • Eight years of experience in technical program management, engineering management, or infrastructure engineering.
  • Experience with production cloud infrastructure at scale, such as distributed systems, Kubernetes, or GPU-backed services.
  • Experience running programs for AI inference or model serving in production, including latency and throughput targets under live customer traffic.
  • Excellent written communication skills and the ability to explain technical details clearly to people outside the work, including customers.
  • Bachelor's degree in Engineering, Computer Science, or a related field.
  • At least four years of program management or related work experience. A relevant advanced degree may substitute for up to two years: a master's degree for one year and a doctorate for two years.

Nice To Haves

  • Ability to reason about where a serving system loses performance or money and distinguish a real regression from noise.
  • Experience with GPU-fleet capacity planning, including forecasting, utilization, and vendor commitments.
  • Experience coordinating work across multiple hardware vendors or cloud providers.
  • Experience setting planning and reporting cadences for an engineering organization.

Responsibilities

  • Partner with engineering and product leaders to scope projects, set milestones, and sequence work according to dependencies and available staffing.
  • Establish and run planning, status-reporting, and risk-review cadences across Cloud Inference and dependent teams.
  • Drive execution from model or engine changes through production traffic, including bring-up, readiness gates, staged rollout, and rollback.
  • Coordinate customer production deployments and migrations end to end, including capacity, cutover plans, shadow traffic, and communications.
  • Define and report serving-fleet health and efficiency metrics, including throughput per GPU, cost per million tokens, latency against customer targets, and deployment lead time; use these metrics to prioritize investment.
  • Run compute capacity planning with infrastructure and finance partners by forecasting demand across clusters and hardware vendors, tracking utilization against commitments, and identifying shortfalls before they block launches.
  • Partner with Security and Infrastructure to ensure deployment-path integrity, including production-cluster access control, secrets handling, and artifact provenance.
  • Improve production reliability by leading post-incident follow-through, driving cross-team root-cause analysis, and ensuring action items are completed.

Benefits

  • Comprehensive healthcare coverage
  • Retirement and savings programs
  • Employee stock purchase opportunities
  • Paid time off
  • Wellbeing resources
  • Family support programs
  • Learning and development opportunities
  • RSU grants
  • Annual target bonus
  • Regular team onsites and local meetups
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service