Principal Clinical AI Evaluation Specialist - MN Preferred

UnitedHealth Group•Minnetonka, MN

2d•$112,700 - $193,200•Remote

About The Position

This Principal-level role serves as a senior technical evaluation and AI governance expert within a multidisciplinary team supporting the UHC Clinical AI Governance Council. This individual contributor role is responsible for ensuring that AI-enabled clinical products and use cases are appropriately tested, evaluated, and documented prior to implementation and that evidence presented to the Council is rigorous, reproducible, and aligned with clinical risks, performance metrics, AI safety considerations, and defined success thresholds. The position owns the evaluation framework and evidence standards used to support AI vendor vetting and clinical AI review. The role ensures that each use case is evaluated against clearly defined risks, metrics, and thresholds to enable objective and consistent Council decision-making. The role advises on and reviews evaluation strategies for clinical AI use cases and, when needed, conducts independent assessments to validate performance claims, testing results, and evidence submitted for Council review. The role serves as the primary technical and methodological liaison among Council leadership, program consultants, and subject matter experts across clinical, data science, engineering, Legal, and Compliance domains. The position provides independent evaluation of AI-enabled solutions and translates complex AI concepts and evidence into clear, decision-ready recommendations for senior clinical leadership.

Requirements

5+ years of experience working with healthcare data, healthcare technology, clinical operations, healthcare analytics, or related healthcare domains
5+ years of experience using Python, SQL, R, or similar tools to develop, test, evaluate, or validate analytic, machine learning, or AI-enabled solutions
5+ years of experience working across clinical, technical, operational, data science, or product teams to translate complex technical concepts into business or clinical recommendations
3+ years of hands-on experience developing, implementing, testing, or evaluating AI solutions, including predictive models, NLP systems, LLM-based applications, chatbots, retrieval-augmented generation (RAG) systems, or agentic workflows
3+ years of experience designing, conducting, or overseeing testing, validation, benchmarking, or performance assessment of AI-enabled solutions, including evaluation of accuracy, safety, reliability, bias, limitations, or clinical appropriateness

Nice To Haves

Experience developing or deploying LLM-based applications, chatbots, retrieval-augmented generation (RAG) systems, agentic AI workflows, or AI-enabled clinical decision-support tools
Experience evaluating AI solutions in healthcare settings, including assessment of accuracy, safety, bias, reliability, workflow integration, or clinical appropriateness
Experience supporting AI governance, model risk management, responsible AI, algorithm oversight, or technology review committees
Experience conducting vendor assessments, proof-of-concept testing, pilot evaluations, or technical due diligence for AI-enabled products
Experience presenting complex technical findings to senior leaders, clinicians, or governance bodies and translating results into actionable recommendations
Experience reviewing, designing, or overseeing test plans for AI-enabled products in healthcare or other regulated environments

Responsibilities

Partner with Council leadership and program consultants to apply the submission framework to each use case, ensuring analytic components align the AI’s intended use and defined risks, metrics, and thresholds
Define use case-specific measurable performance, success criteria, translating clinical risks into measurable metrics and thresholds for decision-making
Ensure complete and clearly defined evaluation methods at intake, including data, populations, comparators, metrics, test methods, and study design
Serve as the primary technical liaison across data science, clinical, and engineering SMEs, synthesizing inputs into a consistent evaluation approach
Ensure that evaluation results support clear decisions against defined performance and success thresholds
Develop concise, decision-ready summaries of performance against defined thresholds, including key limitations of the AI supported program and AI evaluation methodology
Serve as a senior methodological resource during Council review, addressing questions on evaluation design and results
Integrate analytic, clinical, and operational inputs into a single, aligned recommendation
Define requirements for post-deployment monitoring, including metrics, thresholds, accountable governance groups, and triggers for re-review
Design and execute analytic evaluations of clinical AI use cases using real-world healthcare data
Perform hands-on data programming, analysis, and interpretation to support Council reviews and broader evaluation initiatives