Senior Staff, Product Manager- AI Evaluation & Quality

ServiceNow•Santa Clara, CA

23h•$190,900 - $334,100•Hybrid

About The Position

This role sits within APEX (AI Foundations), the team building the platform-level infrastructure behind ServiceNow's enterprise AI strategy. You'll work alongside engineering, data, and applied research partners to make evaluation a durable, compounding advantage. Agentic AI is only as trustworthy as the evaluation discipline behind it. ServiceNow is building the evaluation methodology, tooling, and closed-loop production system that will define how good is good enough for AI specialists and multi-agent frameworks across the enterprise — and we're looking for the product leader to help own that discipline. This is a rare chance to build a category-defining evaluation platform from the ground up, at a company where the outcome shapes how every business unit ships AI, not just one team's roadmap. Evaluation is the foundational differentiator for enterprise agentic AI. As frontier models commoditize, defensible advantage shifts to the orchestration and quality layer — how reliably an AI specialist performs in a customer's production environment. This role sits at exactly that inflection point: You set direction in undefined space — there is no existing playbook to inherit, and the discipline you build becomes the standard others build on. Your impact is horizontal and enterprise-wide by design, driven through influence, tooling, and a Center of Excellence rather than headcount. You'll be the definitive point of reference for AI specialist quality across the portfolio — deep technical and methodological ownership without the dilution of people management. Success is visible and concrete: your framework adopted across business units, a closed-loop system demonstrably improving specialists release-over-release, and tooling used self-serve by teams you never directly staffed.

Requirements

12+ years of software product management experience
Deep experience in AI/ML product management, ideally with hands-on exposure to LLM evaluation, agentic systems, or applied ML quality frameworks.
A track record of defining methodology or standards in ambiguous, cross-team problem spaces — not just executing an existing roadmap.
Strong technical fluency — comfortable in the details of evaluation pipelines, data platforms, and production telemetry, not just the product narrative around them.
Proven ability to drive outcomes through influence and technical credibility, without formal authority over the teams you're working with.

Responsibilities

Define the end-to-end evaluation methodology across ServiceNow's AI specialist portfolio — golden datasets, LLM-as-judge calibration, failure taxonomy, and a multi-layer metric model spanning agent behavior, user experience, and business impact.
Move evaluation from a one-time release gate to a continuous improvement engine, where production telemetry feeds failure analysis, targeted evaluation expansion, and redeployment — making live signal, not synthetic testing, the primary driver of quality.
Partner with a federated Center of Excellence model — central methodology and tooling, embedded practice in each business unit — acting as the internal consulting function that helps teams stand up evaluation without rebuilding infrastructure from scratch.
Drive evaluation from bespoke effort to reusable, self-serve platform capability — evaluation infrastructure, data tooling, and calibration systems that make closed-loop evaluation available to every team building AI specialists.