Product Manager, Agentic AI

Kinetic•Remote ,

1d•Remote

About The Position

Kinetic Insurance is seeking a Product Manager to lead its Agentic AI efforts across the insurance lifecycle. The role ensures that agents are developed as components of a coherent platform, deciding what should be built as an agent, how it should be evaluated, what level of autonomy it may have, and how its impact on business metrics will be demonstrated. The position is more technical than business-focused, but is not an engineering role: the Product Manager will write and test prompts, build evaluation sets, review traces, and prototype ideas without shipping production code. The Product Manager will own Kinetic's agent platform, including orchestration, tooling, observability, and guardrails, and co-own agent solutions with experience-team Product Managers. The role reports to the VP of Product and works with Product Managers leading Broker, Policyholder, Underwriter, Claims, Data, and Business Systems teams. The work includes owning the roadmap for agent orchestration; maintaining an agent registry; defining deployment, versioning, and rollback practices; monitoring usage, cost, and latency; and reviewing existing AI scripts and workflows for retirement, replatforming, or sunset. The role will also partner on use cases, specify agent inputs, tools, autonomy boundaries, escalation paths, success metrics, and baselines, and develop prompts and structured outputs alongside agent and data engineers. The Product Manager will build evaluation datasets and regression suites, establish rollout gates based on evaluation thresholds, maintain autonomy-tier policy and required human checkpoints, and produce supporting documentation such as architecture maps, evaluation histories, audit trails, and before-and-after metrics.

Requirements

5+ years in Product Management, Technical Product Management, or a similar role.
At least 1 year shipping LLM-powered or agentic products into production, beyond demos.
Hands-on prompt engineering experience, including system prompts, tool definitions, multi-step chains, and structured outputs.
Experience building or running evaluations for LLM systems, such as golden datasets, LLM-as-judge graders, and regression testing.
Working knowledge of agent frameworks and platforms such as CrewAI, LangGraph, OpenAI Agents SDK, or AWS Bedrock AgentCore, as well as RAG, tool calling, and MCP.
Ability to convert messy business needs into requirements, user stories, and acceptance criteria.
Startup or high-growth environment experience and comfort building from scratch.
Excellent communication skills, including the ability to explain probabilistic systems to business leaders.
Must reside in the United States and have existing authorization to work in the United States.
Must complete in-person verification of government-issued identity and employment eligibility prior to employment.
Must complete the company's identity and background verification process.

Nice To Haves

Prior experience at an insurtech, MGA, carrier, or another regulated industry such as financial services or healthcare.
Familiarity with workers' compensation, commercial lines underwriting, or claims workflows.
Experience with AI governance frameworks.
Experience with multi-model routing, cost optimization, or human-in-the-loop design for agents.

Responsibilities

Own the roadmap for the agent orchestration layer, including models and routing, tools and memory, observability, permissions, and guardrails.
Run the agent registry, ensuring each agent has one owner, documented purpose, autonomy tier, and location where its behavior is logged.
Define how agents are deployed, versioned, and rolled back, and manage prompts, tool definitions, and model versions like code.
Track usage, cost, and latency for each agent, including cost per orchestration and cost per token.
Inventory AI-related scripts and workflows, then retire, replatform, or sunset them as appropriate.
Keep business logic, specifications, and evaluation data portable across future framework or model changes.
Partner with Product Managers to define problems and determine whether agents or simpler automation are appropriate.
Translate use cases into agent specifications covering inputs, tools, autonomy boundaries, human escalation paths, success metrics, and baselines.
Write, test, and iterate on system prompts, tool descriptions, and structured outputs with agent and data engineers.
Tie each agent to a business metric before scaling it.
Build golden datasets with the data team and run regression suites when prompts, tools, or models change.
Own rollout gates from shadow mode through staged rollout to full production, with go/no-go decisions based on evaluation thresholds.
Write and maintain the autonomy-tier policy, including decisions that require human checkpoints.
Produce architecture maps, evaluation histories, audit trails, and before-and-after metrics.