About The Position

The AIML Data Operations Quality Assurance team develops methods, tools, and metrics to measure quality, catch issues early, and improve how data is produced and evaluated at scale. This hands-on software engineering role builds the tools and data systems that support that work, including automation for QA workflows, pipelines that turn annotation and review events into reliable metrics, and dashboards and alerts that help teams act on those metrics. The role also analyzes quality data to identify where errors occur and assess whether changes to guidelines, tools, or workflows improve outcomes. The engineer owns projects from problem definition through deployment and ongoing operation. Projects may begin with questions about increasing annotation errors or audit-sample coverage. The engineer works with QA, Data Science, Engineering, and Operations to define measurements, build workflows or pipelines, validate results, and deliver them to the people who can act on them.

Requirements

  • Bachelor's degree in Computer Science, Data Science, Engineering, or a related field.
  • Experience designing, building, and maintaining production software, internal tools, or data systems used by other teams.
  • Strong programming skills in Python.
  • Strong SQL skills for data transformation, validation, and analysis.
  • Experience building automated workflows or data pipelines that integrate multiple data sources or services.
  • Experience presenting data through dashboards, analytical applications, or reporting interfaces that support operational decisions.
  • Ability to investigate data discrepancies and translate ambiguous quality or operational questions into measurable outcomes.
  • Ability to work across technical and operational teams and explain findings and tradeoffs clearly.

Nice To Haves

  • Experience with data annotation, human evaluation, ML data pipelines, or quality engineering for data-intensive products.
  • Familiarity with statistical sampling, confidence intervals, inter-annotator agreement, or measurement design for subjective tasks.
  • Experience evaluating rule-based or model-assisted QA against human judgments, including precision, recall, and error analysis.
  • Experience building systems for global operations or workflows with multiple sites and review teams.

Responsibilities

  • Build tools and data systems for quality assurance work.
  • Automate QA workflows.
  • Build pipelines that convert annotation and review events into reliable metrics.
  • Create dashboards and alerts that help teams act on quality metrics.
  • Analyze quality data to identify where errors occur and evaluate whether changes to guidelines, tools, or workflows improve outcomes.
  • Own projects from problem definition through deployment and ongoing operation.
  • Collaborate with QA, Data Science, Engineering, and Operations to define measurements, build workflows or pipelines, validate results, and deliver results to decision-makers.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service