Sr. Model Engineer, Mid-training

xAI•Palo Alto, CA

1d•$180,000 - $440,000

About The Position

SpaceXSI is building Super Intelligence to help humanity understand the universe. Its teams are developing the full Super Intelligence stack, from Colossus supercomputers and Grok models to applications including Grok Bot, Grok Build, Grok Imagine, Grok Voice, and X. The company is creating a cycle in which increased compute enables better models, better models power more useful products, and product usage informs future development. The Sr. Model Engineer, Mid-training will work on scaling synthetic coding data, distilling flagship-model capabilities into flash models, optimizing mid-training data mixtures, developing long-context data recipes, and building evaluations for mid-training checkpoints.

Requirements

Experience writing production-quality code in Python, PyTorch, JAX, Rust, or C++.
Experience training large models, such as pre-training, post-training, fine-tuning, or reinforcement learning training of neural networks with 1B+ parameters.

Nice To Haves

Familiarity with scaling laws across a range of applications.
Strong ability to design machine-learning experiments.
Familiarity with state-of-the-art techniques for curating AI training data for text, image, audio, and video modalities.
Strong engineering abilities in large-scale data processing frameworks.
Experience with machine learning and large-model scaling.
Experience with Spark, Ray, and other large-scale data processing frameworks.

Responsibilities

Scale synthetic coding data to trillions of tokens with large-scale Docker verification.
Distill the intelligence of flagship models into flash models through synthetic data generation.
Optimize mid-training data mixtures to boost the ceiling for reinforcement learning.
Engineer long-context data recipes.
Develop robust and diverse evaluations for mid-training checkpoints.