Product Manager, APEX

Mercor•San Francisco, CA

3d•$180,000 - $300,000•Onsite

About The Position

Mercor's AI Productivity Index (APEX) assesses how effectively frontier AI models can perform economically valuable work. We're looking for a Product Manager to own and scale the APEX brand and public leaderboard. In this role, you will define the positioning, strategy, roadmap, and operational excellence of Mercor's evaluation products, benchmarks, and public and private leaderboards. You will work across Research, Engineering, Operations, and Go-To-Market teams to transform evaluation datasets into trusted industry benchmarks that influence model development and purchasing decisions across the AI ecosystem. You will serve as the product owner for APEX and our eval platform, driving benchmark innovation, evaluation integrity, infrastructure, customer adoption, and business impact. You will work directly with frontier AI labs and enterprise customers, representing Mercor as a thought leader in AI evaluation and measurement. You will partner with Mercor’s world-class benchmark research team, which includes the first authors from many popular benchmarks including Tau Bench, SciCode, PostTrainBench, and more. The ideal candidate combines strong product judgment, technical fluency, operational rigor, and customer-facing experience, with a passion for turning emerging model capabilities into a credible measure of what AI can actually do for the economy.

Requirements

5+ years in product management, technical program management, or a customer-facing technical role.
Prior background in SWE, ML, or DS strongly preferred.
You reason fluently about rubric design, inter-rater reliability, agentic harnesses, contamination and overfitting, and what a small sample can and can't support. You can tell a real capability gap from measurement noise.
You'll publish numbers about other people's models. You know how to be neutral, precise, and defensible under scrutiny.
Strong instincts for which capabilities are actually worth measuring, and what results are actually worth highlighting.
A track record of building a product, program, or business line from nothing.
You take full accountability for outcomes, not just outputs.
Able to self-direct in ambiguous contexts, creating clarity for others.
You can bring research, engineering, operations, and GTM around a shared methodology.

Responsibilities

Own the roadmap and portfolio strategy: Set and influence priorities across new benchmark development, leaderboard launches, infrastructure investment, and expansion into new evaluation categories. Decide which domains earn a slot, when a benchmark has saturated, and what replaces it. Identify opportunity areas for strategic partnership.
Run the intake for new benchmarks: Evaluate and prioritize proposals from research, customers, and GTM against real demand and company strategy.
Enforce eval integrity: Own contamination policy, holdout strategy, versioning, auditability, and release cadence. Publish methodology clearly enough that a skeptical researcher can reconstruct our results.
Build end-to-end eval pipeline: Own the path from eval run to published result, including harness execution, grading, model onboarding, hyperparameter scaffolds, cost and latency reporting, and the leaderboard surface itself.
Work directly with labs and customers: Understand evaluation needs, present and defend results, and turn what you hear into the next benchmark. Support GTM on launches, partnerships, and thought leadership to influence customer model development strategies.
Close the loop to the business: Connect leaderboard demand signals to loss analysis investments and dataset production. Track adoption, usage, and downstream revenue to continuously justify leaderboard ROI.
Get in the weeds: Write specs and PRDs, but also read trajectories, spot-check failures, and make small PRs to unblock yourself and continuously improve the system.