Quality Engineering & AI Validation Manager
Shyft6•New York, NY
•Remote
About The Position
We are seeking a Quality Engineering & AI Validation Manager to lead testing, validation, and quality strategy for AI-driven applications and data platforms. This role will be responsible for ensuring the reliability, accuracy, and performance of solutions built across AWS, Azure, BI tools (Tableau, Power BI), and CRM platforms such as DealCloud. This individual will play a critical role in establishing QA frameworks for AI/ML systems, including model validation, data quality assurance, and end-to-end testing of integrated solutions.
Requirements
- 7+ years of experience in Quality Engineering, QA, or Test Management roles
- Experience leading QA efforts in cloud-based environments (AWS and/or Azure)
- Strong background in test automation frameworks (e.g., Selenium, Cypress, Playwright, or similar)
- Experience testing data pipelines, ETL processes, and data platforms
- Understanding of AI/ML validation concepts, including model testing, data drift, and bias detection
- Experience with API testing and microservices architectures
- Familiarity with BI tools such as Tableau and Power BI
- Strong experience with SQL and data validation techniques
- Experience integrating QA into CI/CD pipelines (DevOps practices)
- Ability to build QA processes in a greenfield or evolving AI environment
- Strong leadership and ability to influence cross-functional teams
- Detail-oriented with a focus on data integrity and system reliability
- Strategic thinker who can balance speed and quality in AI development
Responsibilities
- Lead the QA strategy and testing frameworks for AI-enabled applications and data platforms
- Define and implement AI/ML validation methodologies, including model accuracy, bias detection, and performance testing
- Oversee end-to-end testing across cloud environments (AWS, Azure), data pipelines, BI tools, and CRM integrations
- Establish automated testing frameworks for APIs, data workflows, and application layers
- Develop and enforce data quality validation processes, including completeness, consistency, and lineage checks
- Partner with data, engineering, and architecture teams to ensure testability and quality by design
- Validate integrations across systems including Tableau, Power BI, and DealCloud CRM
- Drive performance, load, and scalability testing for high-volume data and AI workloads
- Implement CI/CD quality gates and integrate testing into DevOps pipelines
- Lead and mentor QA engineers and automation testers
- Ensure compliance with security, regulatory, and governance standards
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Upload and Match Resume
What This Job Offers
Job Type
Full-time
Career Level
Manager
Education Level
No Education Listed
Number of Employees
1-10 employees
