Member of Technical Staff, Mid-training

Hark • San Jose, CA

11h • $180,000 - $450,000

About The Position

Hark is an artificial intelligence company developing proactive, multimodal intelligence that interacts with people and the real world through speech, text, vision, and persistent memory. The company is pairing multimodal models with next-generation AI hardware to create a unified interface between humans and machines.

The Omni team is building AI experiences beyond text, with models that understand and generate content across modalities such as text and audio. The role focuses on building multimodal foundation models and developing the mid-training stages that adapt pretrained models before post-training. This includes creating high-quality and synthetic data mixtures, learning-rate schedules, context extension, and distillation; running experiments to improve model capabilities; and partnering with evaluation and post-training researchers to assess checkpoint quality.

Requirements

• Experience with continued pretraining or late-stage training, such as learning-rate decay, high-quality data mixtures, or context extension.
• Experience building synthetic data pipelines, including generation, verification, and filtering, and evaluating their effects on model quality and data diversity.
• Strong experimental judgment, including isolating recipe changes in controlled ablations and assessing whether improvements transfer to larger runs.
• Experience implementing and debugging distributed training workloads across GPU clusters.
• Experience designing evaluations and identifying confounds such as benchmark contamination, prompt sensitivity, and formatting effects.
• Strong engineering skills, including implementing training methods and building reliable data and experimentation pipelines.
• A record of improving model capabilities or training methods through research, datasets, systems, or production work.

Nice To Haves

• Experience with distillation, rejection sampling, or model-based data generation for reasoning, code, or math.
• Experience with long-context training, including rotary position embedding (RoPE) scaling, context parallelism, or long-document datasets.
• Experience with post-training and evaluating how pretrained checkpoints respond to subsequent adaptation.
• Experience training or distilling smaller models for constrained inference budgets.
• Experience with audio, speech, image, video, or interleaved multimodal datasets.
• Experience detecting benchmark contamination and removing overlapping examples from training data.

Responsibilities

• Develop mid-training recipes using high-quality and synthetic data, including learning-rate decay schedules and data-mixture changes.
• Build synthetic data pipelines with code execution tests, math-answer checks, and filters for reasoning traces.
• Distill capabilities from larger models into smaller models through synthetic data generation and rejection sampling.
• Develop long-context data recipes, including document packing and sequence construction, and evaluate their effects on long- and short-context performance.
• Evaluate mid-training checkpoints with the post-training team, measure how well results predict subsequent model quality, and select checkpoints for further training.
• Run controlled experiments on learning-rate schedules, mixture weights, and data ordering across text, code, and multimodal training data.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed