QA/Test Engineer

Peraton•Basking Ridge, NJ

3d•$86,000 - $138,000

About The Position

Peraton Labs is seeking a QA/Test Engineer to develop and maintain testing frameworks that ensure the accuracy, reliability, and consistency of a generative AI-powered analytical platform's outputs and functionality. This role focuses on the unique challenges of testing generative AI-driven systems, including non-deterministic outputs, analytical accuracy validation, and regression testing for AI behavior changes.

Requirements

Minimum of 5+ years of experience with a Bachelor's degree, 3+ years with a Master's degree, or a PhD in Computer Science or related field
Hands-on QA/test engineering experience
Strong Python proficiency; experience with testing frameworks (pytest, unittest)
Experience with automated testing for data-intensive applications
Understanding of statistical methods for evaluating non-deterministic system outputs
Experience with API testing, integration testing, and end-to-end testing
Familiarity with CI/CD integration for automated test execution
Strong analytical skills and attention to detail
US Citizenship with the ability to obtain/maintain a Secret clearance

Nice To Haves

Experience testing generative AI/ML systems or NLP pipelines
Experience with performance testing tools (Locust, k6, or similar)
Knowledge of security testing methodologies

Responsibilities

Design and implement automated testing frameworks for generative AI agent outputs, including accuracy benchmarks and regression detection
Build validation frameworks for analytical calculations
Develop end-to-end test suites for multi-worker analytical workflows
Implement data quality testing for ingestion pipelines and data fusion outputs
Create test datasets and golden reference outputs for analytical features
Build performance testing frameworks for platform scalability assessment
Develop security testing procedures for sandbox enforcement and access controls
Establish testing standards and quality gates for the CI/CD pipeline