AI Quality & Validation Specialist

PingWind•Annandale, VA

3d•Hybrid

About The Position

PingWind is seeking an AI Quality & Validation Specialist to support the testing, validation, and continuous improvement of applications and AI-enabled solutions prior to release. This junior-level role is ideal for a detail-oriented professional who can evaluate system behavior, identify defects or quality issues, and provide structured feedback to improve application performance and AI-driven outputs. The AI Quality & Validation Specialist will help validate software functionality, assess the quality and relevance of AI-generated results, document findings, and support scenario-based testing from the end-user perspective. This role works closely with technical teams, product stakeholders, and project leads to help ensure released capabilities meet functional requirements and quality standards.

Requirements

1–3 years of relevant experience in software testing, QA support, AI output evaluation, digital content review, or an Associate’s / Bachelor’s Degree in a related field
Strong attention to detail with the ability to identify inconsistencies, defects, or quality issues
Experience documenting findings clearly and communicating actionable feedback
Ability to follow structured instructions, evaluation criteria, and testing procedures
Strong written and verbal communication skills
Comfortable working across multiple tasks in a fast-paced environment
Ability to think from the end-user perspective and test systems against expected outcomes
Basic familiarity with modern software applications, web-based systems, and common collaboration/documentation tools

Nice To Haves

Experience evaluating generative AI outputs such as text, images, audio, or video
Experience supporting software QA, test execution, UAT, or validation activities
Familiarity with defect tracking tools, test case management, or Agile delivery environments
Experience comparing outputs against prompts, business rules, or reference materials
Exposure to prompt testing, AI workflow review, or human-in-the-loop evaluation processes
Experience in a federal contracting environment
Active clearance or eligibility to obtain required clearance, if applicable

Responsibilities

Support functional, regression, and user-focused testing of applications and AI-enabled features before release
Evaluate AI-generated outputs for quality, relevance, accuracy, consistency, and alignment to prompts, instructions, or reference materials
Identify, document, and track software defects, content issues, edge cases, and output anomalies
Perform scenario-based testing by interacting with systems as an end user to validate workflows and expected outcomes
Flag issues in AI or application behavior and provide clear written feedback on what should be corrected or improved
Assist with validation of prompts, workflows, procedures, and output-review criteria used in AI-supported processes
Help review text, image, audio, video, or other generated outputs, as applicable to the program
Maintain test evidence, validation notes, issue logs, and supporting documentation in accordance with project standards
Collaborate with developers, analysts, engineers, and project leadership to clarify expected behavior and support issue resolution
Participate in pilot testing of new features, experimental capabilities, and process improvements
Support continuous quality improvement efforts by identifying recurring issues, trends, and opportunities for better testing coverage
Contribute to the development of repeatable validation procedures, checklists, and quality review practices