Product Manager, Evals

Exa•San Francisco, CA

2d•$160,000 - $300,000•Onsite

About The Position

Exa is hiring a Product Manager for its Evals team, which trains foundational search models and develops ways to measure search quality. The role partners with research, product, go-to-market, and engineering to oversee the quality and completeness of evaluation data. The Product Manager ensures search models are evaluated using comprehensive, high-quality datasets across numerous domains, manages external vendor relationships, and makes strategic decisions about which data to acquire. The work supports Exa’s goal of building highly effective search over the world’s information.

Requirements

Skill in negotiating, securing, and managing relationships with external vendors.
Strong writing skills and care for excellence in written communication; Exa values human communications free of AI-generated text.
High ownership and attention to detail.
Strong numerical literacy and technical background, such as a CS/EE degree, production engineering experience, or equivalent depth earned on the job.

Responsibilities

Build specific, localized evaluation feedback and signal.
Write a manifesto defining what perfect search means.
Define metrics and success criteria for evaluation data suites.
Partner with ML researchers, data engineers, infrastructure engineers, and product teams to shape data loops that improve search models.
Manage relationships with external vendors and negotiate, secure, and acquire data for evaluations.
Oversee the quality and completeness of evaluation data and ensure models are evaluated against comprehensive, high-quality datasets across numerous domains.