Member of Technical Staff, Multimodal Pretraining Data

Hark • San Jose, CA

2h • $180,000 - $450,000

About The Position

As part of Hark’s Omni team, the Member of Technical Staff will develop text, code, image, and video datasets to pretrain Hark’s models, focusing on language and visual understanding. The role involves identifying useful data sources, building filtering and synthetic supervision methods, designing training mixtures, and measuring the impact of data choices through controlled experiments. The successful candidate will work with infrastructure engineers to scale effective methods and deliver datasets that training systems can load efficiently and reproducibly.

Requirements

• Experience developing training data for language or vision-language models, with depth in one or more areas such as text and code curation, image-text datasets, or video data.
• Experience designing controlled data experiments, measuring model improvements, and assessing whether results transfer across training scales.
• Strong Python engineering skills and familiarity with distributed processing frameworks such as Spark, Ray, Beam, or similar systems.
• Understanding of data quality issues such as duplication, noisy supervision, source imbalance, and benchmark contamination.
• A record of improving training data or model performance through research, datasets, systems, open-source contributions, or production work.

Nice To Haves

• Experience with image-text alignment, interleaved document construction, video frame sampling, or temporal grounding.
• Experience with synthetic captioning, model-based filtering, or embedding-based data selection.
• Experience with data mixtures, curricula, or scaling laws for data.
• Familiarity with tokenizer design, visual representations, or training language and vision-language models.
• Experience with provenance, privacy filtering, licensing metadata, or multimodal benchmark decontamination.

Responsibilities

• Develop methods to select, filter, deduplicate, and prepare text, code, image, and video datasets at scale.
• Build paired and interleaved datasets, including image-text documents and video-text data, preserving document structure and alignment between visual content and text.
• Build quality classifiers and sampling methods, and measure their impact through controlled training experiments.
• Develop synthetic captioning and other supervision methods for image and video data, and assess their accuracy, coverage, and biases.
• Design mixtures and curricula across sources, domains, and modalities; evaluate how sampling, sequence construction, and media resolution affect model quality and training efficiency.
• Measure corpus quality and coverage, check for benchmark contamination with evaluation researchers, and track source provenance and dataset versions.
• Work with infrastructure engineers to scale data processing and deliver datasets that training systems can load efficiently and reproducibly.

What This Job Offers

• Job Type: Full-time
• Education Level: No Education Listed