Senior AI Quality & Reliability Engineer

Vizient•Chicago, IL

1d•$102,400 - $179,000

About The Position

Drive the hands-on design, development, and execution of AI Quality Engineering initiatives that support Vizient’s enterprise AI transformation. Design and implement AI Quality Engineering practices, AI validation processes, AI-assisted testing approaches, runtime quality controls, and scalable testing frameworks for responsible deployment of AI-powered business solutions. The role combines hands-on quality engineering, AI-enabled testing modernization, healthcare workflow validation, technical mentorship, and cross-functional collaboration. It advances quality engineering beyond traditional software testing toward AI-native validation, AI-assisted testing, runtime observability, reliability engineering, and modern AI Quality Engineering practices.

Requirements

At least 5 years of experience in Quality Engineering, Quality Assurance, software testing, enterprise application delivery, technology operations, or related technology functions.
At least 2 years of experience providing technical leadership for testing initiatives, automation programs, or enterprise technology delivery projects.
Experience supporting Quality Engineering or Quality Assurance across enterprise platforms, APIs, healthcare applications, operational workflows, or integrated business systems.
Strong knowledge of SDLC, Agile methodologies, test automation frameworks, defect management, release validation, and production support processes.
Strong analytical, problem-solving, organizational, communication, collaboration, and leadership skills.
Demonstrated ability to manage multiple priorities and deliver results in fast-paced, highly collaborative enterprise environments.

Nice To Haves

Relevant degree preferred; an advanced degree in Computer Science, Engineering, or a related field is highly preferred. The posting also specifies a bachelor's degree in Computer Science.
Experience validating AI-powered applications, intelligent automation, machine learning, LLMs, or AI-enabled business workflows.
Experience with AI Quality Engineering practices, AI-assisted testing, runtime observability, monitoring, telemetry, or reliability engineering.
Experience in healthcare technology, payer operations, clinical workflows, or other regulated industries supporting AI governance and responsible AI deployment.

Responsibilities

Design, develop, and execute AI Quality Engineering strategies for AI-powered applications, LLM solutions, intelligent automation, agentic systems, and enterprise AI platforms.
Build and implement scalable AI Quality Engineering practices, including AI-native testing approaches, validation processes, runtime quality controls, reusable testing accelerators, and automated testing frameworks.
Lead AI validation activities, including functional testing, prompt validation, workflow testing, regression testing, release validation, runtime quality assurance, and production reliability support.
Partner with AI Engineering, AIOps, LLMOps, Security, Governance, Clinical, Data, and Product teams to deliver scalable AI Quality Engineering processes across enterprise AI initiatives.
Support runtime reliability through observability, telemetry, distributed tracing, monitoring, drift detection, incident response, and operational quality assurance for AI-enabled systems.
Develop and maintain AI evaluation frameworks, validation datasets, quality scoring methodologies, and automated testing workflows to improve AI solution reliability and scalability.
Collaborate with Clinical, Operational, and Engineering stakeholders to validate healthcare workflows, payer operations, and AI-enabled business processes, supporting responsible AI deployment through human-in-the-loop validation.
Coordinate testing across Agile delivery teams, including sprint planning, test execution, defect management, issue tracking, release readiness, risk identification, and production support.
Mentor Quality Engineers and provide technical guidance that promotes engineering excellence, AI-enabled testing modernization, continuous improvement, and adoption of modern Quality Engineering practices.
Research, evaluate, and recommend emerging AI Quality Engineering, testing automation, observability, and runtime assurance technologies to improve enterprise AI quality capabilities.