Senior Product Manager (AI)

Weekday•San Francisco, CA

About The Position

The Senior Product Manager (AI) owns the strategy, roadmap, and execution of AI product capabilities, with a focus on reliable, autonomous agentic systems for enterprise data operations. The role defines how AI agents detect, investigate, triage, and resolve data incidents, including the context available to agents, actions they may safely perform, how outputs are evaluated, and where human oversight is required. This hands-on product leadership role spans context engineering, resolution and triage agents, and MCP-based integrations with enterprise tools such as Microsoft Teams, Slack, ServiceNow, and Jira. The product manager works closely with engineering and Go-to-Market teams to shape, build, evaluate, launch, and improve AI agents. They translate customer needs, technical constraints, and evolving model capabilities into precise product requirements, while balancing reliability, cost, latency, quality, security, transparency, and user experience. The role also establishes evaluation frameworks and success metrics, investigates product failures, and ensures that agent decisions and actions are understandable, explainable, and correctable for data engineers and analysts.

Requirements

Hands-on experience building and deploying LLM-powered applications or agentic workflows in a technical capacity.
At least 3 years of Product Management experience working on technically complex products such as AI/ML platforms, data platforms, developer tools, or APIs.
Strong working knowledge of foundation models, their capabilities and limitations, and cost, latency, and quality trade-offs.
Understanding of agent architecture, including tool calling, context assembly, retrieval, orchestration, and guardrails.
Experience with prompt engineering, context engineering, and evaluating model performance.
Strong understanding of evaluation methodologies for non-deterministic AI systems.
Ability to identify where AI can deliver meaningful value and where conventional software or human intervention is more appropriate.
Proven ability to work closely with engineering teams and make informed technical and product trade-offs.
Excellent communication skills, including the ability to explain complex technical concepts to technical and non-technical stakeholders.
Bachelor's degree in Computer Science, Data Science, Information Systems, Engineering, or a related technical discipline, or equivalent practical experience.

Nice To Haves

Experience working with MCP, agent frameworks, or integrating AI capabilities into third-party platforms such as Teams, Slack, ServiceNow, and Jira.
Previous ownership of an AI or agent-based product within a B2B SaaS or data-focused organisation.
Familiarity with data observability, data lineage, governance, metadata management, or related platforms.
Hands-on experience in data engineering or data architecture, or experience developing AI solutions on enterprise data systems.
Experience working in a high-growth SaaS, technology, or developer-focused product environment.

Responsibilities

Own the AI product strategy and roadmap, identifying workflows agents can independently manage, assist with, or should not handle.
Evaluate the technical feasibility, failure modes, scalability, and long-term maintainability of agentic product features.
Define agent autonomy boundaries, including which actions agents can perform independently and which require human approval or intervention.
Translate customer requirements, technical constraints, and evolving model capabilities into detailed product requirement documents (PRDs).
Partner with engineering teams to design, develop, evaluate, and launch AI agents that meet standards for accuracy, latency, cost efficiency, observability, and security.
Build and manage the AI evaluation framework, including golden datasets, quality rubrics, and regression testing.
Recreate real-world customer data environments and incident scenarios to validate agent performance and production readiness.
Define and monitor AI product success metrics, including resolution accuracy, acceptance rates, time to root cause, incident deflection, and token cost per outcome.
Analyse product performance, diagnose issues, and improve agent behaviour using data, customer feedback, and evaluation results.
Collaborate with Go-to-Market teams on product launches, technical enablement, and customer-facing documentation.
Ensure users can understand the context, reasoning, and evidence behind agent recommendations and actions.
Stay current on foundation models, agent frameworks, MCP, tool-calling standards, retrieval techniques, and AI evaluation methodologies.
Champion a trustworthy and intuitive AI experience for data engineers and analysts, keeping AI capabilities transparent, explainable, and correctable at scale.