Principal Product Management Engineer

Cisco•Milpitas, CA
•$233,900 - $330,400

About The Position

The Principal Product Management Engineer will lead the strategy, architecture, and organizational adoption of Cisco’s end-to-end AI evaluation platform. The platform includes offline benchmarking suites, automated LLM judges, and continuous production evaluation systems that help teams deploy AI agents and LLM-powered systems safely, reliably, and with verified accuracy at enterprise scale. The role translates quality and safety standards—including factual accuracy, hallucination mitigation, safety policies, and agentic tool-use fidelity—into repeatable test suites and human-in-the-loop review pipelines. It establishes pre-deployment quality bars, risk-based release criteria, and automated regression-detection loops, while connecting offline evaluations with live telemetry, product analytics, and real-time user feedback. The role partners with research scientists, engineering leaders, and product managers across business units to promote standardized evaluation frameworks and organization-wide platform adoption.

Requirements

  • Bachelor's degree in Computer Science, Engineering, Mathematics, or a related technical field and 15 or more years of product management experience; or a master's degree and 12 or more years of experience; or a PhD and 8 or more years of experience.
  • Prior product management leadership experience with AI/ML platforms, model evaluation frameworks, LLM infrastructure, or data-intensive software products.
  • Experience designing and operationalizing offline benchmarking systems, automated evaluation judges, or real-time production quality monitoring pipelines.
  • Experience defining risk-based launch criteria, release-gating frameworks, and regression mitigation workflows for software or AI systems.
  • Experience driving platform adoption across matrixed engineering and product organizations.

Nice To Haves

  • Direct experience evaluating Generative AI, RAG pipelines, and autonomous multi-agent systems.
  • Familiarity with AI evaluation toolkits such as DeepEval, Ragas, LangSmith, or TruLens, and enterprise telemetry architectures.
  • Prior experience in enterprise SaaS, cloud infrastructure, or hyperscaler environments.
  • Background in statistical evaluation methodologies, automated LLM-as-a-judge calibration, and human-in-the-loop annotation workflows.
  • Practical experience with online evaluations for agent systems, including observing production traces and trajectories to diagnose failure patterns and determine improvement techniques.

Responsibilities

  • Define the product vision, staged roadmap, and platform architecture for unified offline benchmarking, automated LLM judges, and continuous production evaluation systems.
  • Convert AI quality and safety standards into objective test suites and structured human-in-the-loop review pipelines.
  • Establish pre-deployment quality bars, risk-based release criteria, and automated regression-detection loops for AI model and application rollouts.
  • Connect offline model evaluations with live telemetry, product analytics, and real-time user feedback to detect and address performance regressions.
  • Drive adoption of standardized evaluation frameworks across business units in partnership with research scientists, engineering leaders, and product managers.

Benefits

  • Medical insurance
  • Dental insurance
  • Vision insurance
  • 401(k) plan with Cisco matching contribution
  • Paid parental leave
  • Short- and long-term disability coverage
  • Basic life insurance
  • Paid holidays and floating holiday
  • Paid birthday day off
  • Paid year-end holiday shutdown
  • Paid personal wellness days
  • Vacation time or flexible vacation time off, depending on employee classification
  • Paid sick time
  • Additional paid time away may be requested for critical or emergency family issues
  • Optional paid volunteer days
  • Annual bonuses for eligible non-sales roles
  • Potential grants of Cisco restricted stock units
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service