Principal Product Manager, AI Infrastructure and Orchestration

DataRobot•San Francisco, CA

20h

About The Position

DataRobot is hiring a Principal Product Manager to own its AI infrastructure and orchestration control plane. The platform deploys agents and models for enterprises that cannot move workloads to public cloud, including regulated industries and sovereign, air-gapped, or customer-managed environments. The control plane determines how workloads are placed, scaled, isolated, routed, and removed across Kubernetes clusters and heterogeneous accelerators, both on DataRobot’s cloud and in customer environments. The role owns the workload and deployment API, placement and capacity management, scaling, agent runtime, traffic and connectivity, governance and audit, metering and packaging, and reliability. The product manager will hold the roadmap for this layer across two engineering pods and align with product teams building on top of it. The position has no direct reports; its scope is technical surface area and influence across engineering pods. This role focuses on workload and agent runtime orchestration: placement, lifecycle, scaling, and isolation of agent and model workloads on Kubernetes and accelerators. It does not own agent coordination and authoring, workflow orchestration in the Airflow or Temporal sense, model quality and applied research, or cluster fabric tuning for distributed training. DataRobot uses AI tooling daily and expects the person in this role to use prototypes, evaluation harnesses, agents connected to real services, or other tools to explore product questions. Candidates are encouraged to bring one or two things they built themselves; production-grade code is not required. DataRobot describes its compensation and benefits as competitive, with employee well-being at the core. Benefits may vary depending on location and local legal requirements. The company is an Equal Employment Opportunity and Affirmative Action employer and is committed to providing reasonable accommodations to applicants with disabilities.

Requirements

At least 6 years of product management experience in infrastructure, developer platforms, or cloud services.
At least 3 years of experience with Kubernetes-based or distributed-systems products.
Principal candidates should have at least 9 years of experience and experience owning a platform layer used by other product teams.
Deep technical understanding of GPUs and accelerators, including topology-aware placement, fractional and time-sliced sharing, MIG, device plugins, driver and container-runtime plumbing, memory constraints, and GPU utilization costs.
Deep technical understanding of Kubernetes, including the API server and scheduler, controllers and CRDs, operators, admission and RBAC, device plugins, resource requests and limits, node pools, and unschedulable pods.
Experience with multi-tenancy, including isolation models, noisy neighbors, quota and fairness, and tenancy designs that withstand customer security review.
Experience owning a public or platform API and understanding the consequences of the API contract shipped.
Ability to use technical writing and prototyping to make a case, such as engineering-facing documentation, deep dives, public posts, API references, or working prototypes.
Comfort operating with matrixed engineering teams and without direct reports.
BS or MS in Computer Science or a closely related technical field, or equivalent hands-on experience as a software, platform, or infrastructure engineer.

Nice To Haves

Depth in service networking, including ingress and routing, load balancing under uneven request cost, DNS, TLS termination, private-link connectivity, and network policy as a tenancy boundary.
Experience with modern serving stacks and their failure modes, such as vLLM or similar systems, KV cache behavior, batching, and quantization trade-offs.
Familiarity with long-running and agentic workload patterns, including session affinity, statefulness, tool-call fan-out, and sandboxed execution.
Experience with customer-managed, air-gapped, or sovereign deployments and associated compliance constraints.
Regulated-industry experience where access control, secrets, and invocation-level auditability are product requirements.
CNCF or open-source contribution.

Responsibilities

Own the deployment and workload API, including its resource model, lifecycle semantics, versioning, backward compatibility, and customer-facing error behavior.
Own workload placement and capacity policy across nodes and accelerators, including quota, priority, and behavior under contention.
Define scaling behavior, including autoscaling signals, cold starts, scale-to-zero economics, headroom policy, and customer-facing cost-versus-latency controls.
Own the agent runtime product, including execution location and duration, isolation, tool-call execution, and state survival across restarts or evictions.
Own traffic and connectivity for model and agent endpoints, including ingress, routing, request-aware load balancing, tenancy boundaries, and private connectivity to customer networks.
Own governance and audit capabilities, including records of deployments and invocations, applicable policy and access control, and audit record retention.
Own metering and packaging, including inference measurement, quota, tenant attribution, and pricing.
Own reliability requirements, including SLOs, error budgets, and operational tools for diagnosing degraded deployments.
Hold the roadmap for the orchestration control plane across two engineering pods and align with product teams building on it.
Use technical writing, prototypes, and AI tooling to explore product questions and make product cases.