Sr AI Quality & Reliability Engineer

TalentOla • Chicago, TX

1d • Hybrid

About The Position

The Sr AI Quality & Reliability Engineer will drive the hands-on design, development, and execution of AI Quality Engineering initiatives supporting enterprise AI transformation efforts. The role will design and implement AI Quality Engineering practices, AI validation processes, AI-assisted testing approaches, runtime quality controls, and scalable testing frameworks supporting responsible deployment of AI-powered business solutions. This role combines hands-on quality engineering, AI-enabled testing modernization, healthcare workflow validation, technical mentorship, and cross-functional collaboration. This role will help advance Quality Engineering capabilities beyond traditional software testing practices toward AI-native validation, AI-assisted testing, runtime observability, reliability engineering, and modern AI quality engineering practices.

Requirements

• Engineering Degree – BE/ME/BTech/MTech/BSc/MSc.

Nice To Haves

• Technical certification in multiple technologies is desirable.

Responsibilities

• Design and execute AI Quality Engineering activities supporting AI-powered applications, LLM-enabled workflows, intelligent automation solutions, agentic systems, and enterprise AI platforms.
• Build and implement AI Quality Engineering practices, including AI-native testing approaches, validation processes, runtime quality controls, reusable testing accelerators, and scalable testing workflows.
• Drive modernization of traditional Quality Engineering practices to address AI-enabled workflows, intelligent orchestration, and evolving healthcare operational workflows.
• Support implementation of AI quality standards, validation approaches, release readiness processes, and governance-aligned testing practices supporting enterprise AI adoption.
• Coordinate AI validation activities, defect management, test execution, release support, and quality improvement initiatives across AI-enabled systems.
• Support AI validation activities, including functional validation, prompt testing, workflow testing, regression testing, runtime quality assurance, and production reliability support.
• Partner with AI Engineering, AIOps, LLMOps, Security, Governance, Clinical, and Data teams to support scalable AI Quality Engineering processes across enterprise AI delivery initiatives.
• Support runtime quality and reliability practices, including telemetry alignment, distributed tracing, observability, monitoring coordination, incident support, release validation, and runtime reliability improvement efforts across AI-enabled systems.
• Support AI evaluation frameworks, validation datasets, quality scoring approaches, and automated testing workflows supporting scalable AI Quality Engineering practices.
• Support drift detection, runtime monitoring, and operational quality assurance initiatives across AI-enabled systems.
• Support human-in-the-loop validation processes and operational review workflows supporting responsible AI deployment.
• Drive adoption of AI-assisted testing approaches, intelligent automation, reusable testing accelerators, and scalable test automation practices.
• Support observability initiatives improving reliability, traceability, and confidence across AI-enabled systems.
• Collaborate with Clinical, Operational, and Engineering stakeholders to support validation of healthcare workflows, payer operations, and AI-enabled business processes.
• Contribute to delivery coordination across AI Quality Engineering initiatives, including sprint execution, testing coordination, issue tracking, risk identification, and release support activities.
• Partner with stakeholders to evaluate testing readiness, implementation dependencies, quality risks, and operational support considerations for AI initiatives.
• Support tooling evaluations, automation initiatives, and modernization efforts supporting AI Quality Engineering maturity.
• Help establish scalable testing processes, reusable quality engineering assets, and operational support models across AI delivery teams.
• Support adoption of modern AI Quality Engineering practices across engineering and delivery organizations.
• Mentor quality engineers and analysts and provide technical guidance to delivery teams while fostering a collaborative, continuously learning, and engineering-focused culture.
• Communicate testing risks, implementation issues, delivery tradeoffs, and operational recommendations to technical and business stakeholders.
• Promote engineering discipline, continuous improvement, responsible AI adoption, and operational accountability across AI Quality Engineering initiatives.
• Research and evaluate emerging AI quality engineering, testing, observability, automation, and runtime assurance technologies supporting continuous improvement.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior