Validation Engineer, AI & Data

RADcube•Indianapolis, IN
•Onsite

About The Position

The Validation Engineer, AI & Data validates the accuracy, reliability, completeness, and consistency of business context and AI-generated responses against approved business requirements and ground truth answers. The role designs and implements testing strategies for domain context models, business context repositories, ontology mappings, and AI response evaluation. It develops representative ground-truth test sets, automated evaluation tests, repeatable scripts, scoring rubrics, regression tests, and accuracy reporting. The engineer documents test evidence, defects, remediation results, AI readiness assessments, and validation outcomes, and supports release readiness, weekly demonstrations, and handover of reusable evaluation frameworks and documentation.

Requirements

  • Expertise in data quality engineering.
  • Expertise in SQL and Python.
  • Expertise in automated testing and PyTest or equivalent frameworks.
  • Expertise in ETL and data validation.
  • Expertise in AI and LLM evaluation.
  • Expertise in ground-truth testing, regression testing, and test automation.

Responsibilities

  • Design and implement testing strategies for the Domain Context Model, Business Context Repository, ontology mappings, and AI response evaluation.
  • Develop representative ground-truth question-and-answer test sets with defined expected results and tolerances, validated by data owners.
  • Build and maintain automated evaluation tests for AI responses generated through the client’s Context Injection toolkit.
  • Evaluate AI responses with and without domain context to establish baseline accuracy and measure improvements.
  • Validate numeric answers, SQL and query results, business logic, aggregations, join relationships, and narrative responses.
  • Identify and classify failures, including hallucinations, stale information, incorrect schemas, faulty joins, missing business context, and persona mismatches.
  • Develop repeatable evaluation scripts, scoring rubrics, regression tests, and accuracy reporting dashboards or scorecards.
  • Verify context repository data completeness, classification consistency, access-control behavior, and changes introduced through automated context updates.
  • Document test evidence, defects, remediation results, AI readiness assessments, and validation outcomes.
  • Support release readiness, weekly demonstrations, and handover of reusable evaluation frameworks and documentation.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service