Product Manager, AI Products

Caseware•Toronto, ON

1d•Hybrid

About The Position

Caseware is seeking a Product Manager to lead the development of AI capabilities within their platform. This role involves defining product strategy, working closely with engineering and domain experts, and ensuring the quality and trustworthiness of AI features for accounting and audit professionals. The position requires high agency, autonomy, intellectual curiosity, and a strong understanding of AI developments relevant to the financial domain. The successful candidate will drive AI initiatives, design evaluation frameworks, translate requirements into product specifications, and collaborate with various stakeholders to define and communicate product direction.

Requirements

4+ years of experience in technology-focused product management or product development.
1-2 years of experience working deeply with AI or machine learning-driven products.
Direct experience building and running evaluation frameworks for AI products, including golden datasets, rubrics, human-in-the-loop validation, and regression tracking.
Technical fluency in LLMs, RAG, prompt engineering, embeddings, and agentic patterns.
High agency: ability to identify needs, act without explicit instruction, and take ownership of outcomes.
Genuine curiosity about AI, including following model releases, reading research, and understanding frontier lab developments.
Experience working with domain experts or subject matter specialists to define quality bars in high-stakes, regulated, or professional-grade contexts.
Comfortable with ambiguity and energized by solving problems without clear playbooks.

Responsibilities

Drive high-impact AI initiatives within the Caseware platform transformation, shaping priorities and sequencing based on user research, domain expertise, and technical feasibility.
Design and run rigorous evaluation frameworks, including building golden datasets, defining task taxonomies, writing rubrics for AI output scoring, and conducting structured evaluation cycles.
Work daily with engineering to translate precise problem statements into well-scoped product requirements with clear acceptance criteria and evaluation bars.
Embed with domain experts and partner accounting and audit firms to validate AI outputs against professional standards.
Drive continuous discovery with practitioners by shadowing workflows, testing prototypes, and identifying specific, actionable failure modes.
Stay ahead of the AI landscape by monitoring model developments, new architectures, emerging agent patterns, and competitor moves, and forming a clear point of view on their relevance to Caseware.
Define and own product health metrics for AI features, such as task completion, correction rates, override frequency, trust signals, and time-to-completion, and use them to drive product decisions.
Synthesize signals from engineering, partners, domain experts, customer success, and leadership into a coherent product direction and communicate it clearly at all organizational levels.