Foundation Model Data Engineer

Sciforium•San Francisco, CA

6h•$155,000 - $210,000

About The Position

Sciforium is seeking a highly technical Foundation Model Data Engineer to lead the strategy, creation, and curation of datasets used to train foundation models. The role owns the end-to-end data lifecycle, from raw web-scale crawling through fine-grained human-alignment datasets, and designs taxonomies, filtering heuristics, and post-training pipelines to support model reasoning, safety, and multimodal understanding. The position focuses on treating data as a high-scale engineering challenge and an analytical discipline. It includes creating pre-training datasets, processing petabyte-scale data, building post-training and alignment datasets, expanding vision and video data capabilities, developing high-throughput Python processing, and analyzing training corpora for biases, knowledge gaps, and quality regressions. Designing synthetic-data pipelines is an added value.

Requirements

At least 5 years of industry experience in Data Science or Machine Learning, with a proven track record of building and managing datasets for foundation models.
Expert-level Python skills, including high-performance code, multiprocessing, multithreading, and efficient memory management for large-scale data tasks.
Demonstrated experience working with petabyte-scale datasets used to train production-grade LLMs or Large Vision Models.
Experience building massive LLM training sets from scratch, including raw web crawls such as Common Crawl and specialized domain data.
Hands-on experience building datasets for RLHF, DPO, and multi-turn instruction following, including managing human-labeling workflows and quality gold sets.
Mastery of data-at-scale frameworks such as Spark or Ray, or high-performance data-loading formats such as WebDataset or Parquet.

Nice To Haves

Experience building large-scale image or video datasets from scratch, such as LAION-style pipelines.
Familiarity with large-scale multimodal data crawling and challenges involving video processing, codecs, and compression.
Experience designing complex labeling schemas for reasoning, coding, and mathematical benchmarks.
A Master’s or PhD in a quantitative field focused on data-centric AI or information retrieval.

Responsibilities

Own the creation of pre-training datasets for LLMs, including defining the mix of web data, code, books, and technical papers.
Design and implement pipelines for cleaning, exact and fuzzy deduplication, and signal extraction from petabytes of unstructured data.
Develop post-training datasets, including supervised fine-tuning instructions, multi-turn dialogues, and preference-modeling data for RLHF and DPO.
Drive acquisition and processing of vision and video data, addressing multimodal alignment, video compression, and temporal data consistency.
Develop high-throughput Python data-processing scripts using multiprocessing and multithreading.
Analyze training corpora statistically to identify biases, knowledge gaps, and quality regressions.
Design pipelines for generating high-reasoning synthetic data to augment natural datasets, using existing models for labeling and refinement.