Product Manager, Model Gateway

Mercor•San Francisco, CA

13h•$180,000 - $300,000•Onsite

About The Position

In this Platform Product Manager, you'll own Mercor's LLM gateway, the shared layer every product surface depends on to reach frontier models reliably, affordably, and within policy. This is a high-leverage platform role: the decisions you make about priority queueing, cost attribution, rate limits, and model usage policy ripple across every team that builds on top of us. You'll own this product area with significant autonomy, working directly with an infrastructure team that owns the underlying shared services and with leadership to turn ambiguous, cross-cutting tradeoffs into durable systems and clear policy. Because the gateway sits between every product team and the models they rely on, your users are internal: the PMs and engineers building on the platform, plus the infra and finance partners who care about reliability and spend. Success looks like product teams moving fast without stepping on each other, model costs that are transparent and attributable down to the team and workload, and usage policies that scale as we add surfaces and models.

Requirements

5+ years as a product manager, ideally on platform, infrastructure, or developer-facing products; prior background in SWE, ML, or DS strongly preferred.
Systems thinking: You reason fluently about queues, rate limits, quotas, and the tradeoffs between latency, throughput, and cost, and you can turn those tradeoffs into policy.
High ownership: You take full accountability for outcomes not just outputs.
Independence: Able to self-direct in ambiguous, cross-cutting contexts.
Technical fluency: Comfortable reading usage data, writing queries, and building scrappy dashboards or making small PRs to validate direction and unblock engineering.
Stakeholder alignment: You can bring many product teams around shared constraints and turn contention over capacity and spend into rules people accept and plan around.
Cost & metrics orientation: Comfortable with cost models, unit economics, and attribution, and equally able to push forward a longer-term platform vision.

Responsibilities

Own the LLM gateway end-to-end: Set the roadmap and policy for the shared layer that routes every product surface to the models it depends on, from priority queueing to cost attribution to model access, and own its lifecycle from conception through execution.
Design priority and fairness: Define the queueing and rate-limiting logic that decides whose workload runs when, balancing revenue blocking traffic against background jobs, and making the tradeoffs explicit and consistent.
Make cost visible and attributable: Work backwards from business outcomes to make product decisions that impact business health and growth.
Negotiate rate limits and usage policy: Turn contention over shared capacity into clear, defensible rules people can plan around.
Partner with infrastructure: Work hand-in-hand with the central infra team translating product needs into platform requirements and platform constraints back into product tradeoffs.
Work backwards from the business: Connect gateway decisions to business health, cost, reliability, and velocity, and use that to prioritize.