Director, AI Quality & Reliability Engineering

Vizient•Chicago, IL

5d•$117,600 - $206,000•Onsite

About The Position

In this role, you will lead operational execution and scaling of enterprise AI Quality Engineering capabilities across Vizient. Help operationalize scalable AI Quality Engineering frameworks, AI validation processes, runtime quality practices, AI-enabled testing approaches, and enterprise quality standards supporting responsible industrialization of AI-powered business solutions at enterprise scale. Combine quality engineering leadership, AI-enabled testing modernization, healthcare workflow validation, operational execution, organizational capability development, and cross-functional delivery leadership.

Requirements

10 + years experience implementing quality engineering practices for Generative AI, Large Language Models (LLMs), agentic AI systems, or intelligent automation solutions.
Relevant degree, Bachelor’s degree preferred.
Experience working within healthcare, payer, provider, life sciences, or other regulated industries.
Familiarity with AI governance, responsible AI frameworks, model risk management, compliance requirements, and auditability practices.
Experience with AI-assisted testing tools, test automation platforms, observability solutions, telemetry platforms, and modern quality engineering technologies.
Knowledge of AIOps, LLMOps, MLOps, cloud-native platforms, and enterprise-scale AI deployment models.
Relevant certifications in Quality Engineering, Software Testing, Agile, Cloud Technologies, AI, or related disciplines.
Experience building and scaling organizational capabilities, operating models, and quality engineering centers of excellence.

Responsibilities

Lead and scale enterprise AI Quality Engineering capabilities supporting AI-powered applications, LLM-enabled workflows, intelligent automation solutions, agentic systems, and enterprise AI platforms.
Establish and operationalize AI-native quality engineering frameworks, validation methodologies, testing standards, runtime assurance practices, and scalable delivery processes.
Drive modernization of traditional quality engineering and test automation practices to support AI-enabled workflows, probabilistic systems, intelligent orchestration, and evolving healthcare business processes.
Lead AI validation strategies, including functional testing, prompt testing, workflow validation, regression testing, release readiness, runtime quality assurance, and production reliability.
Partner with AI Engineering, Data, Security, Governance, Clinical, and Operational teams to ensure quality, reliability, scalability, and successful deployment of enterprise AI solutions.
Implement quality controls, governance-aligned testing practices, validation evidence processes, and risk mitigation strategies that support responsible AI adoption and regulatory compliance.
Advance enterprise observability, telemetry, monitoring, and runtime quality practices to improve traceability, reliability, and confidence across AI-enabled systems.
Lead AI Quality Engineering delivery activities, including organizational planning, vendor and contractor management, execution governance, tooling strategy, and capability development.
Build, mentor, and develop high-performing quality engineering teams while fostering a culture of innovation, accountability, continuous improvement, and engineering excellence.
Research and evaluate emerging AI quality engineering, testing, automation, observability, and runtime assurance technologies to drive continuous improvement and organizational maturity.
Experience building and scaling organizational capabilities, operating models, and quality engineering centers of excellence.