Manager - AI Tech Project Management - Remote

UnitedHealth Group•Eden Prairie, MN

10d•$91,700 - $163,700•Remote

About The Position

Join Optum Technology to lead the product strategy, roadmap, and enterprise adoption of AI Patrol – Evals as a Service. This critical shared capability gives engineering and product teams a standard way to measure the quality, safety, and cost of AI solutions before and after release. In this role, you will convert fragmented, team-by-team evaluation practices into a governed, reusable service with defined intake, standard scorecards, and automated release gates. Working directly inside the AI stack, you will define evaluation dimensions for large language models and agentic architectures, curate gold-standard and adversarial test sets, and implement automated LLM-as-judge scoring to accelerate the path from prototype to production release across the enterprise. You’ll enjoy the flexibility to work remotely from anywhere within the U.S. as you take on some tough challenges. For all hires in the Minneapolis or Washington, D.C. area, you will be required to work in the office a minimum of four days per week.

Requirements

Undergraduate degree or 4+ years of equivalent technical product or program management experience
5+ years of experience in technical product or program management owning product roadmaps, intake, prioritization, and writing measurable acceptance criteria
2+ years of experience managing platform or shared-service technical products
2+ years of experience or working fluency with large language models (LLMs), retrieval-augmented generation (RAG), and agentic architectures
2+ years of experience with AI model and application evaluation methods (eg, gold-set design, red-teaming/adversarial testing, hallucination measurement, or drift monitoring)
1+ years of experience using SQL and Python for reading evaluation telemetry, metrics, and dashboards

Nice To Haves

Undergraduate degree in Computer Science, Data Science, Engineering, or a related technical field
Hands-on experience with MLOps/LLMOps tools and CI/CD release gating concepts
Experience with cloud AI platform services (AWS, Azure, GCP)
Familiarity with healthcare data handling, including PHI and HIPAA compliance obligations
Demonstrated ability to influence without authority across engineering, data science, governance, and business leaders
Executive-level communication skills with experience presenting standardized evidence for AI governance, risk, and compliance stakeholders

Responsibilities

Own the product strategy, roadmap, intake prioritization, and enterprise adoption of AI Patrol – Evals as a Service
Design, develop, and deploy AI-powered evaluation solutions to address complex business challenges with an emphasis on responsible AI use
Convert fragmented evaluation practices into one governed, reusable shared service with standard scorecards and release gates
Define evaluation dimensions for large language model (LLM) and agentic solutions, including accuracy, groundedness, hallucination rate, bias, toxicity, latency, and cost per task
Curate gold-standard and adversarial test sets, set thresholds for automated LLM-as-judge and human-in-the-loop scoring, and prioritize key platform features
Leverage enterprise-approved AI tools to streamline workflows, automate tasks, analyze requirements, summarize telemetry, and produce evaluation reporting
Evaluate emerging trends in LLMOps and model evaluation to inform solution design, benchmarking standards, and strategic platform innovation