QA Validation Engineer for AI Initiatives (Teradyne, North Reading, MA)

Teradyne•North Reading, MA

4d•$126,100 - $201,800•Onsite

About The Position

The QA/Validation Lead Engineer for AI Initiatives is responsible for leading the end-to-end validation, testing, and quality assurance of all AI/ML systems from a data-centric perspective. This role ensures that proper data sources, structures, pipelines, and governance frameworks are validated and maintained across all AI initiatives, including generative AI, agentic systems, and human-in-the-loop workflows. The role combines strategic QA leadership with hands-on technical validation to safeguard the accuracy, reliability, compliance, and ethical use of data powering AI solutions. This position reports to the IT Quality Assurance Manager.

Requirements

Bachelor’s degree in business, information technology or related field
7+ years in QA/validation engineering
2+ years focused on AI/ML or data-intensive systems
Software test automation engineer with hands-on coding experience
Solid understanding of databases to assess structural correctness
Familiarity with regulatory and compliance standards to support or lead audit-related activities
Programming languages such as Python, Java, or similar
SQL
Test automation frameworks
ML metrics
Data profiling tools
Generative AI, agent orchestration, prompt engineering advanced skills
Data lineage, data governance, schema validation, ETL/ELT testing, data lake/warehouse structures
Familiar with AI ethics, risk management, audit processes
Leadership, cross-functional collaboration, change management, technical documentation, stakeholder communication
Advanced skills in data analysis, process optimization, and problem-solving
Exceptional written, verbal, and visual communication skills, with the ability to engage both technical and non-technical audiences
Proven ability to work effectively with stakeholders across multiple levels and departments, including balancing competing priorities (negotiation skills)
Ability to manage ambiguity and shifting priorities in a fast-paced environment

Nice To Haves

Master's in Computer Science, Data Science, Engineering, or related field
Experience with multi-agent testing and API integration validation
Knowledge of AI-human interaction design and workflow optimization
Certifications in AI/ML, data governance, or quality engineering
Experience establishing QA Centers of Excellence for AI programs

Responsibilities

Validate that AI/ML models are consuming accurate, authorized, and properly structured data sources.
Design and execute data quality test strategies to assess completeness, consistency, lineage, and timeliness of training and inference data.
Identify and flag data hallucinations, logic errors, and edge cases in AI model outputs traceable to data issues.
Develop and lead comprehensive test strategies for AI/ML systems, including accuracy, bias, robustness, and regression testing.
Oversee scenario-based testing and output validation for generative AI and LLM-driven applications.
Automate validation suites for agentic/multi-agent systems, integration testing, and CI/CD pipelines for ML models.
Assess AI model performance through both manual and automated review, with emphasis on data quality impact on outputs.
Validate prompt engineering outputs from a data accuracy standpoint, ensuring responses are grounded in verified data sources.
Conduct scenario testing to identify cases where data gaps or structural flaws produce unreliable outputs.
Partner with prompt engineers to optimize data retrieval and grounding strategies.
Ensure all AI data sources and structures meet governance, regulatory, and compliance standards.
Lead audit processes, documentation, and risk assessments related to data usage in AI systems.
Conduct bias detection and explainability testing to ensure ethical and compliant AI use.
Maintain traceability of data lineage for regulatory reporting and audit readiness.
Validate data flows across human-plus-AI workflows and agent orchestration systems.
Ensure seamless data integration across multi-agent architectures and API interfaces.
Collaborate with data engineering, data science, AI/ML engineering, and product teams to establish data quality standards.