Lead Architect, AI Quality Engineering

Vizient•Edina, MN

About The Position

Lead architecture and technical implementation of AI Quality Engineering solutions supporting AI-powered applications, LLM-enabled workflows, intelligent automation solutions, agentic systems, and enterprise AI platforms. Design and implement scalable AI validation frameworks, AI-assisted testing approaches, runtime quality controls, reusable testing accelerators, and workflow optimization capabilities supporting enterprise AI delivery initiatives. Support modernization of traditional Quality Engineering practices through intelligent automation, workflow orchestration, and scalable quality engineering patterns. Develop automated validation approaches, anomaly detection processes, testing pipelines, and quality metrics supporting governance-aligned AI deployment practices. Design and optimize human-in-the-loop validation workflows, operational review processes, and AI quality assurance controls supporting reliable and scalable AI-enabled systems. Partner with engineering and business teams to identify operational bottlenecks, workflow optimization opportunities, automation use cases, and scalable quality engineering improvements. Support AI validation activities, including prompt testing, workflow testing, regression testing, runtime quality assurance, and production reliability support. Partner with AI Engineering, AIOps, LLMOps, Security, Governance, Clinical, and Data teams to support scalable AI Quality Engineering and workflow automation processes across enterprise AI initiatives. Design and support runtime quality practices, including telemetry alignment, monitoring coordination, validation processes, and runtime reliability improvement efforts. Drive adoption of AI-assisted testing approaches, intelligent automation, reusable testing accelerators, and orchestration-aware testing practices. Support observability and runtime visibility initiatives improving reliability, traceability, and confidence across AI-enabled systems. Collaborate with Clinical, Operational, and Engineering stakeholders to support validation of healthcare workflows, operational processes, and AI-enabled business solutions. Support delivery coordination activities across AI Quality Engineering and workflow optimization initiatives, including implementation planning, issue tracking, operational support, and release coordination activities. Partner with stakeholders to evaluate implementation readiness, workflow dependencies, operational risks, automation opportunities, and quality considerations for AI initiatives. Support tooling evaluations, automation frameworks, orchestration tooling, and modernization initiatives supporting AI Quality Engineering maturity. Help establish reusable workflow automation patterns, scalable testing assets, and engineering enablement practices across delivery teams. Support adoption of modern AI Quality Engineering and workflow optimization practices across engineering and business organizations. Lead and mentor engineers, analysts, contractors, and delivery teams while fostering a collaborative, continuously learning, and engineering-focused culture. Communicate implementation risks, workflow optimization opportunities, technical tradeoffs, and operational recommendations to technical and business stakeholders. Promote engineering discipline, continuous improvement, responsible AI adoption, and operational accountability across AI Quality Engineering initiatives. Research and evaluate emerging AI quality engineering, workflow automation, observability, and runtime assurance technologies supporting continuous improvement initiatives.

Requirements

Architecture and technical implementation of AI Quality Engineering solutions
Scalable AI validation frameworks
AI-assisted testing approaches
Runtime quality controls
Reusable testing accelerators
Workflow optimization capabilities
Intelligent automation
Workflow orchestration
Scalable quality engineering patterns
Automated validation approaches
Anomaly detection processes
Testing pipelines
Quality metrics supporting governance-aligned AI deployment practices
Human-in-the-loop validation workflows
Operational review processes
AI quality assurance controls
Prompt testing
Workflow testing
Regression testing
Runtime quality assurance
Production reliability support
AI Engineering, AIOps, LLMOps, Security, Governance, Clinical, and Data teams collaboration
Runtime quality practices, including telemetry alignment, monitoring coordination, validation processes, and runtime reliability improvement efforts
Observability and runtime visibility initiatives
Validation of healthcare workflows, operational processes, and AI-enabled business solutions
Delivery coordination activities, including implementation planning, issue tracking, operational support, and release coordination activities
Evaluation of implementation readiness, workflow dependencies, operational risks, automation opportunities, and quality considerations for AI initiatives
Tooling evaluations, automation frameworks, orchestration tooling, and modernization initiatives supporting AI Quality Engineering maturity
Reusable workflow automation patterns
Scalable testing assets
Engineering enablement practices
Adoption of modern AI Quality Engineering and workflow optimization practices
Leadership and mentoring of engineers, analysts, contractors, and delivery teams
Communication of implementation risks, workflow optimization opportunities, technical tradeoffs, and operational recommendations
Promotion of engineering discipline, continuous improvement, responsible AI adoption, and operational accountability
Research and evaluation of emerging AI quality engineering, workflow automation, observability, and runtime assurance technologies

Responsibilities

Lead architecture and technical implementation of AI Quality Engineering solutions
Design and implement scalable AI validation frameworks, AI-assisted testing approaches, runtime quality controls, reusable testing accelerators, and workflow optimization capabilities
Support modernization of traditional Quality Engineering practices through intelligent automation, workflow orchestration, and scalable quality engineering patterns
Develop automated validation approaches, anomaly detection processes, testing pipelines, and quality metrics
Design and optimize human-in-the-loop validation workflows, operational review processes, and AI quality assurance controls
Partner with engineering and business teams to identify operational bottlenecks, workflow optimization opportunities, automation use cases, and scalable quality engineering improvements
Support AI validation activities, including prompt testing, workflow testing, regression testing, runtime quality assurance, and production reliability support
Partner with AI Engineering, AIOps, LLMOps, Security, Governance, Clinical, and Data teams to support scalable AI Quality Engineering and workflow automation processes
Design and support runtime quality practices, including telemetry alignment, monitoring coordination, validation processes, and runtime reliability improvement efforts
Drive adoption of AI-assisted testing approaches, intelligent automation, reusable testing accelerators, and orchestration-aware testing practices
Support observability and runtime visibility initiatives improving reliability, traceability, and confidence across AI-enabled systems
Collaborate with Clinical, Operational, and Engineering stakeholders to support validation of healthcare workflows, operational processes, and AI-enabled business solutions
Support delivery coordination activities across AI Quality Engineering and workflow optimization initiatives, including implementation planning, issue tracking, operational support, and release coordination activities
Partner with stakeholders to evaluate implementation readiness, workflow dependencies, operational risks, automation opportunities, and quality considerations for AI initiatives
Support tooling evaluations, automation frameworks, orchestration tooling, and modernization initiatives supporting AI Quality Engineering maturity
Help establish reusable workflow automation patterns, scalable testing assets, and engineering enablement practices across delivery teams
Support adoption of modern AI Quality Engineering and workflow optimization practices across engineering and business organizations
Lead and mentor engineers, analysts, contractors, and delivery teams
Communicate implementation risks, workflow optimization opportunities, technical tradeoffs, and operational recommendations to technical and business stakeholders
Promote engineering discipline, continuous improvement, responsible AI adoption, and operational accountability across AI Quality Engineering initiatives
Research and evaluate emerging AI quality engineering, workflow automation, observability, and runtime assurance technologies