Principal Product Manager, AI Infrastructure and Orchestration

DataRobot•Boston, WA

About The Position

DataRobot delivers AI that maximizes impact and minimizes business risk. Our platform and applications integrate into core business processes so teams can develop, deliver, and govern AI at scale. DataRobot empowers practitioners to deliver predictive and generative AI, and enables leaders to secure their AI assets. Organizations worldwide rely on DataRobot for AI that makes sense for their business — today and in the future. We run agents and models in production for enterprises that cannot move their workloads to a public cloud: regulated industries, sovereign deployments, air-gapped and customer-managed clusters. The control plane is the layer that makes that possible. It deploys the agents, it deploys the models those agents call, and it decides how every workload is placed, scaled, isolated, routed to, and torn down across Kubernetes clusters and heterogeneous accelerators, on our cloud and on the customer's. Agents and models are one deployment problem here. An agent is a long-lived workload with session state and unpredictable fan-out, calling models with wildly uneven cost profiles, and both land on the same finite pool of accelerators. Allocating that pool correctly is the job. You will own this layer as a product. You will spend your time in design reviews, in the API contract, and in production data.

Requirements

6+ years in product management for infrastructure, developer platforms, or cloud services, with at least 3 years on Kubernetes-based or distributed systems products.
Principal candidates bring 9+ years and experience with a platform layer that other product teams built on.
Deep technical understanding of GPU and accelerator behavior: topology-aware placement, fractional and time-sliced sharing, MIG, device plugins, driver and container runtime plumbing, memory as a constraint, and utilization costs.
Deep technical understanding of Kubernetes: API server and scheduler, controllers and CRDs, operators, admission and RBAC, device plugins, resource requests and limits, node pools, and scheduling behavior.
Multi-tenancy experience: isolation models, noisy neighbors, quota and fairness, and tenancy designs that pass security reviews.
API product judgment: experience owning a public or platform API and its consequences.
Proficiency in technical writing and prototyping for communication and decision-making.
Comfort operating with matrixed engineering teams and no direct reports.
BS or MS in Computer Science or a closely related technical field, or equivalent hands-on experience as a software, platform, or infrastructure engineer.

Nice To Haves

Service networking depth: ingress and routing, load balancing under uneven request cost, DNS, TLS termination, private link connectivity, and network policy.
Modern serving stacks and their failure modes: vLLM or similar, KV cache behavior, batching, quantization trade-offs.
Long-running and agentic workload patterns: session affinity, statefulness, tool-call fan-out, sandboxed execution.
Experience with customer-managed, air-gapped, or sovereign deployments and associated compliance constraints.
Regulated-industry experience with product requirements for access control, secrets, and invocation-level auditability.
CNCF or open-source contribution.
Experience building prototypes, evaluation harnesses, agents, or tools to exercise APIs or answer roadmap questions.

Responsibilities

Own the deployment and workload API, including resource model, lifecycle semantics, versioning, backward compatibility, and error behavior.
Define placement and capacity strategies for workloads on nodes and accelerators, including quota, priority, and contention management.
Develop scaling strategies, including autoscaling signals, cold start and scale-to-zero economics, and headroom policy.
Oversee agent runtime, including isolation, tool call execution, and state persistence.
Manage traffic and connectivity for model and agent endpoints, including ingress, routing, load balancing, tenancy boundaries, and private connectivity.
Define governance and audit requirements, including deployment tracking, invocation records, policy enforcement, and access control.
Develop metering and packaging strategies for inference measurement, quota attribution, and pricing.
Establish reliability SLOs, error budgets, and diagnostic tools for platform engineers.
Hold the roadmap for the AI infrastructure and orchestration layer across two engineering pods and align with product teams building on top of it.