Research Engineer, QC Automation

Invictus Direct•San Francisco, CA

3d•Onsite

About The Position

Our client is seeking a Research Engineer, QC Automation to automate quality control for training data created by companies using its infrastructure. This role will build systems that scale quality as the company meets continued strong demand, combining technical execution with strong judgment about data quality and genuine curiosity about unfamiliar domains.

Requirements

Proficiency in Python, Docker, and Linux environments
Strong judgment about what good data means and how to measure it
Genuine curiosity about unfamiliar domains and skill at asking the questions needed to understand them
Experience building scalable data-validation pipelines or automated QA/QC systems without a prescribed roadmap
Experience with benchmarks and evaluations
Early-stage startup experience and independent execution
Technical aptitude and learning potential matter more than years of experience.

Nice To Haves

Knowledge of statistics
Strong written and verbal communication
Comfort designing metrics, experiments, and QA/QC processes
Ability to construct tasks in new evaluations
Comfort in unstructured problem spaces

Responsibilities

Create QC systems grounded in true understanding and human judgment without relying heavily on LLMs
Define and enforce quality standards for training data
Design experiments and metrics to grade agent outputs
Partner with data vendors to diagnose agent failure modes, debug quality issues, and improve data-generation processes
Build systems for auditing supplier datasets, including sampling strategies, rule-based and model-assisted validation pipelines, and feedback loops
Integrate QC learnings into infrastructure tools and the data-vendor portal to reduce anomalies, inconsistencies, and edge cases