Technical AI Quality Coordinator

RWS Group • Colorado Springs, CO

2d

About The Position

The Technical AI Quality Coordinator is an integral part of our Data Services team, blending a strong Quality Assurance background with practical scripting and automation skills. In this role, quality is your primary compass—you will help define our quality strategy, design technical testing frameworks, and evaluate the output quality of our distributed workforce. You will play a key role in ensuring top-notch data integrity across AI annotation, RLHF (Reinforcement Learning from Human Feedback), and model evaluation pipelines through a combination of data analysis, light coding, and programmatic quality design.

Requirements

• 3+ years of experience in Quality Assurance, Data Quality, or AI Operations.
• Working knowledge of Python and SQL for data analysis, reporting, and automation.
• Experience supporting AI/ML data projects, including data annotation, evaluation, or training data programs.
• Strong analytical skills with the ability to identify quality issues, trends, and opportunities for improvement.
• Excellent English communication skills and experience collaborating with global, cross-functional teams.

Nice To Haves

• Degree in Data Analytics, Computer Science, Statistics, Linguistics, Psychology, or a related field.
• Experience with reporting and visualization tools such as Power BI, Tableau, or Looker.
• Familiarity with remote or crowdsourced workforce management.

Responsibilities

• Support quality assurance across AI data projects by improving processes, automating quality checks, and monitoring data accuracy.
• Develop and maintain testing frameworks, evaluation datasets, and quality metrics to ensure consistent project outcomes.
• Investigate quality issues, identify root causes, and implement corrective actions to improve performance.
• Help establish and maintain quality standards, frameworks, and auditing processes across projects.
• Monitor contributor and team performance, providing insights and recommendations to drive quality improvements.
• Partner with cross-functional teams including Data Engineering, AI Engineering, and Solutions teams to embed quality controls and reporting into workflows.
• Track key quality and performance metrics, identify risks, and support continuous improvement initiatives.
• Stay informed on AI industry trends and contribute ideas to enhance quality programs and best practices.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed