Conversational AI QA Engineering Lead Director

CVS Health • Hartford, CT

7d

About The Position

We’re building a world of health around every individual — shaping a more connected, convenient and compassionate health experience. At CVS Health®, you’ll be surrounded by passionate colleagues who care deeply, innovate with purpose, hold ourselves accountable and prioritize safety and quality in everything we do. Join us and be part of something bigger – helping to simplify health care one person, one family and one community at a time. Position Summary We are looking for an exceptionally skilled and driven Conversational AI QA Engineering Leader. The ideal candidate will have a robust background in building and guiding high-performing QA teams focused on conversational AI solutions. This role involves coaching and mentoring QA engineers to ensure the delivery of high-quality voice and chatbot products and services through rigorous testing and validation strategies. If you are passionate about quality assurance in conversational AI and have a proven track record of leading successful QA teams in a fast-paced environment, we would love to hear from you.

Requirements

• 10+ years of experience in QA leadership roles, with at least 5 years focused on cloud-native environments (IBM & Azure highly preferred) testing conversational AI solutions, including voice and chatbots.
• 10+ years of experience translating complex QA strategies into strategic business outcomes.
• 10+ years of experience with QA automation and scripting using languages such as Java, Python, or Node.js.
• 7+ years of experience as a QA lead for large-scale AI or digital transformation programs working in a high-performance Agile environment.
• Strong understanding of conversational AI technologies (e.g., chatbots, voice assistants, LLMs, STT/TTS, NLU/NLP engines) and QA techniques for validating these technologies, including model testing, regression, and performance validation.

Nice To Haves

• Executive-level communication skills, capable of effectively articulating QA and strategic concepts to multiple stakeholders.
• Experience with IBM WatsonX and its suite of AI tools and services.
• Advanced degree (Master’s or PhD) in Computer Science, Engineering, AI, or a related field.
• Hands-on experience with Large Language Models (LLM) and their practical applications in conversational AI QA.
• Familiarity with SAFE agile methodologies and experience in leading agile QA teams.
• 7+ years of experience with API testing tools such as Postman, Swagger, or similar tools.

Responsibilities

• Lead enterprise QA strategy for Conversational AI platforms, ensuring high‑quality, scalable delivery of voice and chatbot solutions aligned with business objectives and customer experience goals.
• Own end‑to‑end quality assurance execution across AI model validation, automation, and manual testing, driving consistent quality outcomes and timely release cycles.
• Build, develop, and inspire high‑performing QA teams, including QA engineers and test managers, fostering a culture of excellence, collaboration, and continuous improvement.
• Establish and evolve QA frameworks, governance, and KPIs to ensure performance, compliance, risk management, and measurable ROI for conversational AI initiatives.
• Champion modern QA practices and emerging technologies, including agile testing methodologies, automation frameworks, and AI‑specific QA tools.
• Partner closely with product, engineering, and leadership teams to identify quality risks, communicate insights and metrics, and proactively improve customer experience.

Benefits

• Affordable medical plan options, a 401(k) plan (including matching company contributions), and an employee stock purchase plan.
• No-cost programs for all colleagues including wellness screenings, tobacco cessation and weight management programs, confidential counseling and financial coaching.
• Benefit solutions that address the different needs and preferences of our colleagues including paid time off, flexible work schedules, family leave, dependent care resources, colleague assistance programs, tuition assistance, retiree medical access and many other benefits depending on eligibility.

What This Job Offers

• Job Type: Full-time
• Career Level: Director