Member of Technical Staff, Pretraining Data

Mirendil•San Francisco, CA

About The Position

Mirendil is a tech-first company focused on solving core bottlenecks that unlock step-change acceleration across science and technology. Its first goal is to democratize frontier AI R&D across scientific disciplines. The company is building a frontier AI research company and training its own models end-to-end. The Pretraining Data role helps determine what the company’s models train on as it scales to larger models. The role uses agents to run ablations, pipelines, and analyses, and works closely with researchers responsible for model architecture and training runs. The position focuses on building and improving pretraining data pipelines, deciding data mixtures, designing experiments to justify those decisions, extending code-first data mixtures into scientific domains, and maintaining corpus quality, provenance, decontamination, and reproducible versions.

Requirements

Have shipped a data change into a large pretraining run and be able to show the ablation that justified it.
Have built data pipelines at the scale of trillions of tokens using Spark, Ray, Beam, or similar.
Write strong Python.
Know PyTorch or JAX well enough to run personal training experiments.
Have hands-on experience with at least two of the following: deduplication, quality classifiers, mixture optimization, data scaling laws, or long-context and code data.
Be able to assess whether experimental results are real and explain reasoning clearly in writing.

Responsibilities

Build pretraining data pipelines, including pipelines for repository-scale code and technical documents.
Improve extraction, filtering, and deduplication of web pages, open-source code, and public datasets.
Decide the data mixture for each training run and design ablations and data-scaling experiments to justify it.
Direct agents to run experiments and pipelines in parallel and assess which results to trust.
Extend a code-first data mixture into scientific domains, starting with areas where the models fall short.
Maintain corpus cleanliness as it grows, including provenance, decontamination against evaluations, and reproducible versions.
Work closely with researchers responsible for model architecture and training runs.