Software Engineer, AI Data & Evaluation

Mercor•San Francisco, CA

1d•$130,000 - $500,000•Onsite

About The Position

As a Senior Software Engineer (AI Data & Evaluation) at Mercor, you will be at the core of building the data infrastructure and evaluation systems that power the next generation of frontier AI models. Our team's mission is to develop high-quality data types that push frontier models forward and drive the AI industry ahead. Software Engineers on this team are builders and innovators first. You will design and develop the evaluation methods and flywheels that drive continuous model improvement, engineer synthetic data pipelines and environments that generate high-signal training data at scale, and build the operational automation that keeps it all running with precision and efficiency. This role demands a product- and impact-oriented mindset, a bias toward shipping, and the ability to thrive at the intersection of data engineering, systems design, and applied AI research.

Requirements

Strong software engineering skills with a proven track record shipping production systems end-to-end.
Deep interest in and experience with AI/ML data pipelines, evaluation frameworks, or training data systems.
Systems thinking: ability to design for scalability, quality, and operational reliability simultaneously.
Comfort operating with ownership and pragmatism in fast-moving, ambiguous environments.
Effective communication and collaboration with engineering, research, and operations teams.

Nice To Haves

Experience with synthetic data generation, reinforcement learning environments, or large-scale data quality systems is highly valued.

Responsibilities

Innovate and develop evaluation methodologies and flywheels that continuously improve data quality and model performance at scale.
Design and build synthetic data generation systems and simulation environments that produce high-signal, high-diversity training data for frontier AI models.
Architect and ship operational automation systems that maximize throughput, efficiency, and quality across the end-to-end data pipeline.
Collaborate cross-functionally with Operations, Research, and Product to translate evolving model needs into robust, scalable engineering solutions.
Own end-to-end delivery of critical systems — from prototyping novel ideas to scaling production infrastructure.