AI/ML Test Engineer
About The Position
Modern Technology Solutions, Inc. (MTSI) is hiring an AI/ML Test Engineer in Springfield, VA. Qualified candidates must be U.S. citizens with a TS/SCI clearance and CI Polygraph. This role involves executing system-level AI/Machine Learning evaluation activities within the Test & Evaluation (T&E) environment supporting mission-focused operational testing. Responsibilities include performing integration testing, data capture, drift detection, and operational validation activities across AI/ML evaluation workflows. The engineer will support automated and semi-automated evaluation pipelines for AI/ML model testing and performance assessment, and develop, maintain, and refine data annotation workflows supporting operational evaluation requirements. Validation of operational datasets and support for testing activities across multiple models, datasets, and mission scenarios are key. The role also requires conducting integration testing of AI/ML models within operationally representative environments and workflows, adapting evaluation procedures to address evolving GEOINT data conditions and operational mission requirements, and supporting high-throughput and parallel model evaluation activities. Identifying and mitigating data drift, performance degradation, and workflow issues impacting evaluation reliability is crucial. Collaboration with Senior AI/ML Test Engineers, data scientists, software engineers, and mission stakeholders is expected, as is contributing to technical documentation, evaluation reporting, and customer briefings supporting operational decision-making.
Requirements
- 4+ years of experience supporting AI/ML testing, system integration, operational testing, or evaluation workflows.
- Familiarity with automated testing pipelines, operational data validation, and model performance assessment concepts.
- Ability to support multiple evaluation activities across dynamic operational environments.
- Experience troubleshooting technical workflow or data-related issues in support of operational objectives.
- Strong analytical, organizational, and communication skills.
- U.S. citizens with a TS/SCI clearance and CI Polygraph.
Nice To Haves
- Experience supporting DoD, NGA, or Intelligence Community programs.
- Familiarity with AI/ML model evaluation, GEOINT workflows, and operational testing environments.
- Experience working in Agile, DevSecOps, or cloud-based operational environments.
- Familiarity with annotation workflows, data capture pipelines, drift detection, and automated evaluation tools.
- Experience supporting scalable AI/ML evaluation or mission-focused T&E activities.
Responsibilities
- Execute system-level AI/Machine Learning evaluation activities within the Test & Evaluation (T&E) environment supporting mission-focused operational testing.
- Perform integration testing, data capture, drift detection, and operational validation activities across AI/ML evaluation workflows.
- Support automated and semi-automated evaluation pipelines for AI/ML model testing and performance assessment.
- Develop, maintain, and refine data annotation workflows supporting operational evaluation requirements.
- Validate operational datasets and support testing activities across multiple models, datasets, and mission scenarios.
- Conduct integration testing of AI/ML models within operationally representative environments and workflows.
- Adapt evaluation procedures and workflows to address evolving GEOINT data conditions and operational mission requirements.
- Support high-throughput and parallel model evaluation activities to maintain alignment with mission timelines and operational priorities.
- Identify and mitigate data drift, performance degradation, and workflow issues impacting evaluation reliability.
- Collaborate with Senior AI/ML Test Engineers, data scientists, software engineers, and mission stakeholders to support scalable T&E execution.
- Contribute to technical documentation, evaluation reporting, and customer briefings supporting operational decision-making.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
