Research Engineer

talentpluto•San Francisco, CA

10h•Onsite

About The Position

Our partner is hiring a Research Engineer to help scale the quality assurance (QA) systems behind training data generated through their infrastructure. This role sits at the intersection of data quality, tooling, and applied ML operations: you’ll build the standards, pipelines, and feedback loops that ensure datasets are reliable, consistent, and ready for training and evaluation. You’ll work closely with internal stakeholders and external data suppliers to diagnose quality issues, improve workflows, and continuously fold QA learnings back into the platform. If you enjoy building systems that make high-quality data scalable—and want to do it in a high-ownership, fast-paced environment—this role is a strong fit.

Requirements

Proficiency with Python and experience working in Linux environments
Experience with Docker and reproducible development/deployment workflows
Experience working with large-scale datasets (validation, transformation, or analysis)
Strong problem-solving skills and evidence of rapid learning in technical environments
Ability to operate independently and deliver results in an early-stage, fast-moving setting
Clear written and verbal communication skills (including collaborating across time zones)

Nice To Haves

Experience building data validation pipelines and/or human-in-the-loop review systems
Familiarity with common training-data failure modes and techniques to detect subtle inconsistencies
Comfort designing QA metrics, experiments, and processes—not just executing predefined checks
Familiarity with modern AI tooling and LLM capabilities

Responsibilities

Define and enforce quality standards for training datasets used for RL training and evaluation
Build tooling and workflows to audit supplier-generated datasets, including sampling strategies, validation pipelines (rule-based and model-assisted), and feedback loops
Evaluate and implement human-in-the-loop review workflows where beneficial to improve quality and efficiency
Partner with external data suppliers to debug quality issues, provide actionable feedback, and improve their data generation processes
Integrate QA learnings into internal tools and supplier portals to reduce anomalies, inconsistencies, and edge cases over time
Track QA outcomes and continuously improve processes, metrics, and documentation