AI Governance & Evaluation Analyst

UT Southwestern Medical Center•Dallas, TX

About The Position

The AI Governance & Evaluation Analyst supports UT Southwestern’s AI governance program by managing the review, evaluation, and monitoring of AI use cases across healthcare, research, education, and administrative functions. The analyst works with project teams, subject matter experts, governance committees, and institutional partners to ensure AI systems are reviewed, documented, monitored, and deployed responsibly. The role coordinates AI case intake and review, evaluation and assurance activities, governance committee processes, vendor oversight, and governance for generative and agentic AI. It also provides stakeholder guidance and support, monitors governance metrics, prepares reports for leadership, and helps improve governance procedures and materials.

Requirements

Master’s degree in Public Health, Biomedical Informatics, Data Science, Computer Science, Engineering, Research Administration, or a related field.
Five years of professional experience in AI/ML, healthcare operations, clinical informatics, research administration, data analytics, technology governance, compliance, quality, program management, or a related area.
Experience coordinating complex projects, reviews, or governance processes involving multiple stakeholders.
Strong organizational, project coordination, and documentation skills.
Ability to communicate complex technical and regulatory information clearly.
Strong relationship-building, facilitation, and customer service skills.
Ability to manage multiple priorities and deadlines.
Working knowledge of AI technologies, including machine learning and generative AI.
Ability to handle confidential information with discretion.

Nice To Haves

PhD or other advanced degree in a related field.
Experience in healthcare, academic medicine, research, life sciences, or other regulated environments.
Experience with AI governance, AI evaluation, validation, assurance, vendor review, digital health, or lifecycle monitoring.
Familiarity with responsible AI frameworks and standards such as NIST AI RMF and ISO 42001.

Responsibilities

Manage intake, triage, tracking, and documentation of AI use cases.
Guide teams through governance requirements and review processes.
Coordinate reviews, approvals, monitoring activities, and reassessments.
Maintain accurate, complete, and audit-ready records and escalate high-risk or complex issues when appropriate.
Coordinate AI evaluations using approved standards and methodologies; collect and organize evidence concerning performance, safety, fairness, privacy, security, and reliability.
Support validation, testing, monitoring, and assurance activities; document findings, mitigation plans, and monitoring requirements.
Track post-deployment performance and report significant issues.
Prepare agendas, case summaries, and review materials for governance committees; coordinate meetings, reviewers, and subject matter experts.
Document committee decisions, action items, and follow-up activities; communicate outcomes and track required actions.
Maintain records of AI systems, vendors, approvals, risk classifications, and monitoring requirements.
Coordinate AI-related vendor due diligence with relevant departments and monitor system changes to ensure reassessment when necessary.
Maintain templates, evidence repositories, and governance documentation.
Support governance activities for large language models, foundation models, multimodal AI, and agentic AI systems.
Review information about model performance, oversight, access controls, guardrails, and testing; track model changes and document limitations and monitoring requirements.
Serve as a primary point of contact for assigned AI initiatives and collaborate with clinical, research, technical, compliance, legal, privacy, and operational teams.
Provide guidance, training, consultations, and office-hour support and promote consistent governance practices across the organization.
Monitor governance metrics, workload, review timelines, and stakeholder experience.
Prepare dashboards, reports, and presentations for leadership.
Identify process improvements and support the development of procedures, templates, and training materials.
Stay informed about emerging AI technologies, regulations, and industry standards.

Benefits

PPO medical plan, available day one at no cost for full-time employee-only coverage.
100% coverage for preventive healthcare with no copay.
Paid Time Off, available day one.
Retirement programs through the Teacher Retirement System of Texas (TRS).
Paid Parental Leave Benefit.
Wellness programs.
Tuition reimbursement.
Public Service Loan Forgiveness (PSLF) Qualified Employer.
On-site childcare.
Paid holidays.
Merit increases.