Member of Technical Staff, Pretraining Architecture and Scaling
About The Position
Hark is an artificial intelligence company developing proactive, multimodal intelligence that interacts through speech, text, vision, and persistent memory, alongside next-generation AI hardware. The Omni team is building AI experiences beyond text and aims to create seamless, real-time multimodal intelligence across modalities including text and audio. As a member of the Omni team, you will build multimodal foundation models and work across the pretraining stack, including data curation and mixture design, model architecture, scaling, distributed training, and evaluation. You will research architectures and training methods, design experiments to understand how results change with model and dataset scale, and use findings to improve model quality and training efficiency. The role also involves implementing research ideas and developing training recipes in collaboration with data researchers and training engineers.
Requirements
- Hands-on experience training large models and making trade-offs among model quality, compute cost, and training stability.
- Strong experimental judgment, including designing controlled experiments that distinguish between competing hypotheses.
- Experience applying scaling laws to experiment design and evaluating extrapolations from smaller training runs.
- Experience implementing and debugging distributed training workloads across GPU clusters.
- Strong engineering skills, including implementing research ideas, profiling training workloads, and debugging model behavior.
- A record of improving model quality or training efficiency through research, engineering, open-source contributions, or production work.
Nice To Haves
- Experience with mixture-of-experts routing, sparse architectures, or long-context methods.
- Work on data mixtures, curriculum, or tokenizer design.
- Kernel-level optimization or mixed-precision training experience.
- Experience with efficiency work aimed at constrained inference targets, including on-device.
Responsibilities
- Conduct research on model architecture, optimization, and scaling to improve the capability and efficiency of large models.
- Establish baselines and run controlled experiments to evaluate architectural and optimization changes at larger scales.
- Develop training recipes, including learning-rate schedules, context lengths, token budgets, and compute allocation.
- Investigate mixture-of-experts architectures, hybrid attention, and long-context training support.
- Diagnose loss spikes, divergence, and numerical issues, working with training engineers to distinguish modeling problems from infrastructure failures.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
