Member of Technical Staff, Pretraining Data

Mirendil

2d•$300,000 - $400,000

About The Position

Mirendil is a tech-first company focused on solving core bottlenecks that unlock step-change acceleration across science and technology. Its first goal is to democratize frontier AI R&D across scientific disciplines. The company is building a frontier AI research company and training its own models end-to-end. The role focuses on deciding what data Mirendil’s models train on as the company scales to larger models. The person in this role will use agents to run ablations, pipelines, and analyses, and will work closely with researchers responsible for model architecture and training runs.

Requirements

Have shipped a data change into a large pretraining run and be able to show the ablation that justified it.
Have built data pipelines at the scale of trillions of tokens using Spark, Ray, Beam, or similar tools.
Write strong Python.
Know PyTorch or JAX well enough to run training experiments independently.
Have hands-on experience with at least two of the following: deduplication, quality classifiers, mixture optimization, data scaling laws, or long-context and code data.
Be able to judge whether experimental results are reliable and explain reasoning clearly in writing.

Responsibilities

Build pretraining data pipelines, including pipelines for repository-scale code and technical documents.
Improve the use of public data through extraction, filtering, and deduplication of web pages, open-source code, and public datasets.
Decide the data mixture for each training run and design ablations and data-scaling experiments to justify those decisions.
Direct agents to run experiments and pipelines in parallel, and assess which results to trust.
Extend a code-first data mixture into scientific domains, starting with areas where the models fall short.
Maintain corpus quality as it grows, including provenance, decontamination against evaluations, and reproducible versions.