Product Operations Manager - AI

Rec Technologies•San Francisco, CA

5d•Onsite

About The Position

Rec is using AI to reimagine how the world plays, participates in community, and gets active. Our vision is simple: more recreation for everyone. We sit at a rare intersection — a consumer product loved by everyday players and an enterprise platform trusted by the organizations that power recreation. The Product Operations Manager - AI will run the quality engine behind Rec's AI agent, Seb. This role involves managing evals, annotation, and defect triage, and acting as the connective tissue that keeps AI initiatives moving across product, engineering, and the business. The individual will analyze eval results, manage feedback collection, and ensure the AI agent's quality at scale.

Requirements

5+ years in product operations, program management, or a related operational role — ideally with exposure to AI/ML products.
Comfortable getting hands-on with evals, annotation tools, and quality frameworks, even without a formal ML background.
Highly organized — you can run multiple workstreams at once without losing the thread.
A clear communicator who can translate messy technical detail into a simple status update or decision.
Genuinely curious about AI and how agents actually behave in production.
Comfortable in ambiguity — you build the process where none exists yet.
Bias toward action and high attention to detail in equal measure.

Responsibilities

Run Seb's evaluation and quality operations — building and maintaining eval sets, managing annotation queues, and classifying issues as defects vs. preferences.
Own the feedback loop between real-world agent performance and the roadmap — turning quality signals into clear, prioritized fixes.
Build and refine the operational infrastructure (observability, scoring, review workflows) that lets the team trust Seb's quality at scale.
Own our adoption KRs - understand why people use Seb, and why they don't.
Spot patterns across evals and staff feedback that point to bigger product or trust issues before they become customer-facing problems.
Keep AI leadership and stakeholders clearly informed on program status, quality trends, and risk.