Product Manager, Toast IQ Evaluation

Toast

7d • $125,000 - $254,000 • Hybrid

About The Position

As Product Manager, Toast IQ Evaluation, you will define what constitutes useful help for restaurant operators and build evidence to determine whether Toast IQ delivers it. You will study real usage, work directly with operators, and convert findings into labeled datasets, quality standards, and improvement priorities. Your work will help Toast teams understand where the assistant succeeds, where it falls short, and where there are opportunities to improve.

This is a hands-on product role involving regular session review, example labeling, failure investigation, and development of better evaluation processes. Restaurant judgment, attention to detail, and the ability to turn evidence into action are central. You will serve as the team’s first restaurant domain expert, work alongside engineers from a shared prioritized backlog, and report through Product.

Requirements

• Experience working in a restaurant or closely with restaurant operators; understanding decisions involving menus, labor, sales, and inventory in busy service environments.
• Comfort with detailed work, including reading many transcripts, manually labeling examples, and keeping datasets accurate and current.
• Ability to investigate structured data using a database, queries, or AI coding tools and verify that results are trustworthy.
• Precise thinking and writing skills to turn ambiguous observations into clear failure definitions or consistently applicable scoring rules.
• Ability to explain what happened, why it matters to operators, and what should change to engineers, product teams, and leadership.
• Curiosity about AI systems, including how large language model systems fail, and willingness to learn evaluation methods on the job.

Nice To Haves

• Familiarity with Toast products.
• Product management experience is highly valued but not required; experience identifying problems, understanding them, exploring solutions, building, measuring, and iterating under another title is relevant.

Responsibilities

• Review real Toast IQ sessions regularly and describe failures in plain language that reflects operators’ experience.
• Work directly with restaurant operators, including in live sessions, to ensure evaluations reflect what matters to them.
• Group observations into failure modes, measure their frequency, and assess their impact on operators to prioritize problems.
• Create and maintain labeled evaluation examples for common workflows, edge cases, and known failures.
• Validate fixes in real sessions and convert important production failures into regression cases for future releases.
• Lead reviewer calibration sessions and help product teams create and maintain evaluation cases for their capabilities.
• Identify tools that are not delivering value, intents Toast IQ cannot handle, and workflows requiring product team attention; communicate operator problems and impacts clearly.
• Use AI coding tools to build faster review interfaces, automate recurring analysis, monitor common failures, prototype fixes, and improve processes from session review through validation.

Benefits

• Competitive compensation and benefits programs.
• Cash compensation, including overtime and bonus/commissions if eligible.
• Equity.
• Benefits intended to support a healthy lifestyle and flexibility for changing needs.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed