Senior/Staff Product Manager, Market Place

Recruiting From Scratch•San Francisco, NY

3d•Onsite

About The Position

As a Product Manager for the Model Gateway platform, you'll own one of the company's most critical internal products: the shared infrastructure layer that routes every AI request across the organization. You'll define how engineering teams access frontier models, optimize reliability and cost, and establish the policies that allow dozens of product teams to build quickly while efficiently sharing AI infrastructure. This is a highly technical platform product role with broad organizational impact.

Requirements

8+ years of Product Management experience, preferably building platform, infrastructure, developer, or internal-facing products.
Previous experience as a Software Engineer, Machine Learning Engineer, Data Scientist, or other highly technical role strongly preferred.
Strong understanding of distributed systems, platform architecture, APIs, and cloud infrastructure.
Experience working with AI infrastructure, LLM platforms, or large-scale backend systems.
Ability to reason about system tradeoffs involving latency, throughput, rate limits, queues, quotas, and reliability.
Strong analytical skills with experience building dashboards, writing SQL, and using data to drive product decisions.
Proven ability to lead highly cross-functional initiatives spanning multiple engineering organizations.
Excellent written and verbal communication skills with experience aligning technical and business stakeholders.
High ownership mindset with the ability to independently drive ambiguous platform initiatives.

Nice To Haves

Experience building internal developer platforms or infrastructure products.
Familiarity with LLM gateways, model routing, AI inference infrastructure, or distributed compute platforms.
Experience managing infrastructure costs, unit economics, and resource allocation at scale.
Strong understanding of capacity planning, workload prioritization, and platform governance.
Experience balancing technical constraints with business objectives in rapidly growing organizations.
Startup experience building foundational platform capabilities from the ground up.
Comfortable prototyping lightweight technical solutions using AI-assisted development tools.
Passion for AI infrastructure, platform engineering, and enabling high-performing engineering organizations.

Responsibilities

Own the strategy, roadmap, and execution for the company's shared LLM gateway platform.
Define product requirements for model routing, priority scheduling, rate limiting, usage policies, and access controls.
Design scalable systems that balance latency, throughput, reliability, and infrastructure costs across multiple AI workloads.
Build transparent cost attribution and usage reporting that enables teams to understand and optimize model consumption.
Partner closely with Infrastructure Engineering to translate platform constraints into scalable product capabilities.
Develop policies that govern shared AI resources while enabling product teams to move quickly.
Analyze usage patterns, infrastructure metrics, and business outcomes to prioritize platform investments.
Align engineering, finance, infrastructure, and product stakeholders around shared capacity planning and platform governance.
Build dashboards, write queries, and prototype lightweight solutions to validate ideas and accelerate execution.
Continuously evolve the platform as new AI models, workloads, and organizational needs emerge.