Quality Engineering Lead – Agent Evaluation & AI Platforms

Comcast•Mount Laurel, NJ

8d•Onsite

About The Position

We are seeking seasoned Quality Lead (SDET 4) to orchestrate our enterprise release cycles and lead the quality strategy for our next-generation AI initiatives. This role is designed for a strategic technical leader who combines deep Comcast domain expertise across all 4 Lines of Business (LOBs) and Device Provisioning with specialized experience in Agentic AI validations. You will lead the quality process from the front: running scrums, managing the end-to-end defect lifecycle, conducting structural code reviews, and serving as the principal architect for our Agent Evaluation strategies within Google ADK environments. The AI Agentic Team designs, builds, and runs intelligent agents end to end. We own agent logic, workflows, integrations, production deployment, and ongoing performance. By combining build and operations, we deliver agentic systems that are reliable, secure, and scalable in real-world use.

Requirements

10+ years of experience in Software Quality Engineering, with a seasoned background in managing large-scale delivery and leading functional testing strategies.
2+ years of specific experience with Agentic AI validations, Prompt Engineering, and Agent Eval techniques (specifically with Google ADK or similar models).
Deep, demonstrable knowledge of all 4 Comcast LOBs and their underlying technical architectures. (asIS)
Comprehensive understanding of the functional lifecycle for device provisioning, including complex billing system (ACSR and AMDOCS) handoffs and activation workflows.
Strong functional understanding of automation frameworks (e.g., Python, TypeScript), providing the technical insight necessary to oversee code quality and ensure automated suites align with business requirements.
Power-user proficiency in JIRA for advanced dashboarding and Kibana for deep-dive log analytics to drive data-driven quality decisions.
Bachelor's Degree (or equivalent combination of coursework and experience, or extensive related professional experience)

Nice To Haves

Python
TypeScript

Responsibilities

Run daily Scrums and stand-ups, optimizing team velocity, managing task assignments, and ensuring delivery milestones are met on time.
Lead rigorous defect triaging meetings. You will be the primary driver tracking high-priority issues, coordinating directly with development teams, and pushing defects to rapid closure.
Embed yourself in upstream requirement meetings. You will challenge assumptions, identify missing edge cases, and establish the technical "Definition of Done."
Act as the strategic partner to Business teams, leading User Acceptance Testing (UAT) and translating complex technical deployments into business-ready validation.
Bring at least 2+ years of hands-on experience in Agentic AI validations, navigating the non-deterministic nature of large language models and intelligent workflows.
Drive the strategy for Agent Evaluation techniques, ensuring the team effectively implements automated verification patterns (such as LLM-as-a-judge) within Google ADK environments.
Apply deep knowledge of Prompt Engineering to validate agent behavior, test fallback mechanisms, and ensure intent optimization.
Conduct technical Code Reviews for the team's automation repositories. Ensure that Python or TypeScript test scripts targeting the AI layers are modular, efficient, and scale effectively.
Apply deep operational knowledge across all 4 Comcast LOBs (Xfinity Video, High-Speed Data, Digital Voice, and Xfinity Mobile) to ensure cross-functional workflows are flawlessly validated.
Leverage an intricate understanding of backend Device Provisioning pipelines, activation flows, and equipment states to isolate complex environments and system-level issues.
Maintain strong technical oversight of the team's codebase. You will possess a solid understanding of test automation methodologies and basic knowledge of testing frameworks to conduct structural Code Reviews, ensuring the automation scripts meet quality, maintainability, and scalability standards.
Guide junior engineers in designing comprehensive Test Plans and Strategies. You will define what to test and how to test it, ensuring total coverage of complex subscriber journeys.
Navigate Kibana logs and system metrics independently to perform deep-dive root cause analysis on failures before handing them off to development.
Build and manage advanced JIRA dashboards to visualize real-time quality trends, escape-defect rates, and overall sprint health for leadership.
Prepare and present Daily and Weekly Status Reports to stakeholders, providing absolute transparency on risks and release readiness