Sr. Product Manager, AI

Pantomath•San Francisco, CA

1d•$124,000 - $182,000

About The Position

Pantomath is hiring a Senior Product Manager for AI with direct, hands-on experience building agentic systems to own the strategy and execution of our AI product surface. This role sits squarely at the intersection of product leadership and applied AI engineering. You will be responsible for defining how Pantomath's agents resolve data incidents: what context they can see, what actions they are trusted to take, how their output is evaluated, and where the boundary sits between what the agent does and what the human decides. That work spans our context layer (structural, behavioral, baseline, risk, and semantic context), our resolution and triage agents, and our MCP-based distribution into the surfaces customers already work in (Teams, Slack, ServiceNow, Jira). The bar for this role is hands-on. You deeply understand frontier model capabilities and limitations, fluency in the parts that decide whether an agent is trustworthy: what goes into its context and what gets left out, tool and API surfaces the model can actually use correctly, evaluation for non-deterministic systems, and the cost/latency/quality frontier and the unique UX considerations of non-deterministic systems. Enough depth to read a failed trace and name the defect - retrieval, tool definition, context assembly, or prompt. You will use that experience to set technical direction, write precise requirements, and partner closely with engineering to deliver agents that are reliable enough to earn autonomy in production data environments. Pantomath is building the autonomous layer for enterprise Data Operations. Our agents catch data incidents, trace them to root cause across the stack and resolve them, pinpointing their root cause, and resolve them with operational context from across your entire stack. Backed by a $30M Series B led by General Catalyst (alongside Sierra Ventures, Bowery Capital, Epic Ventures, and Hitachi Ventures), Pantomath is pioneering the transition enterprises from passive data monitoring to real-time, autonomous incident resolution. Across our entire company, Pantomath brings together seasoned startup founders, operators with prior successful exits, and veteran talent from market-defining technology companies including Google, Meta, Stripe, Databricks, Microsoft and Amazon. We combine the high-ownership, high-velocity culture of a scaling venture-backed startup with the deep execution rigor and track record of proven industry leaders.

Requirements

Hands-on experience building with LLMs in a technical role (e.g., shipping production AI features, building agent workflows, prompt and context engineering, running model evaluations).
3+ years of product management experience working on technically complex products such as AI/ML platforms, data platforms, developer tools, or APIs.
Strong, working knowledge of: Foundation model capabilities, limitations, and cost/latency trade-offs
Strong, working knowledge of: Agent architecture patterns: tool calling, context assembly, retrieval, orchestration, and guardrails
Strong, working knowledge of: Evaluation methodology for non-deterministic systems
Demonstrated judgment about when AI is the right answer and when it is not.
Proven ability to partner deeply with engineering teams and make informed trade-offs.
Ability to clearly communicate complex technical concepts to both technical and non-technical stakeholders.
Bachelor's degree in Computer Science, Data Science, Information Systems, Engineering, or a related technical field (or equivalent practical experience).

Nice To Haves

Experience with MCP, agent frameworks, or embedding AI into third-party surfaces such as Teams, Slack, ServiceNow, or Jira.
Prior ownership of an AI or agent product within a B2B SaaS or data product.
Familiarity with data observability, lineage, governance, or metadata platforms.
Hands-on background in data engineering or data architecture, or experience building AI on top of enterprise data systems.
Experience in a high-growth SaaS or developer-focused product environment.

Responsibilities

Own the AI product strategy and roadmap, making informed, technical decisions about which workflows an agent should own end to end, which it should assist, and which it should stay out of.
Leverage your applied AI background to evaluate feasibility, failure modes, and long-term maintainability of agentic features.
Define and defend the agent boundary: what the agent is trusted to do autonomously, what requires human confirmation, and how that boundary widens as trust is earned.
Translate customer needs, technical constraints, and model capabilities into clear, detailed PRDs that engineers can execute against without ambiguity.
Partner closely with engineering to design, build, evaluate, and ship agents that meet strict standards for accuracy, latency, cost, observability, and security.
Build and own the evaluation harness: define golden datasets, quality rubrics, and regression testing so that model and prompt changes ship on evidence rather than anecdote.
Replicate real-world customer data environments and incident scenarios to validate agent behavior, and ensure production readiness.
Define success metrics for AI features (resolution accuracy, acceptance rate, time to root cause, deflection, token cost per outcome) and actively monitor, diagnose, and iterate based on data and feedback.
Work with Go-to-Market teams to support launch readiness, technical enablement, and customer-facing documentation, including how to explain what the agent knows and why it is right.
Stay current on trends and best practices across foundation models, agent frameworks, MCP and tool-calling standards, retrieval, and evaluation to inform roadmap decisions.
Advocate for a best-in-class experience for data engineers and analysts, ensuring AI capabilities are transparent, correctable, and trustworthy at scale.