Research Engineer - Distributed Training

Prime Intellect•San Francisco, CA

9d•$150,000 - $350,000•Hybrid

About The Position

Prime Intellect is building the open superintelligence stack, providing infrastructure for AI labs. Their platform, Lab, unifies compute, environments, evaluations, secure sandboxes, high-performance training, and deployment for post-training at frontier scale. They are developing open-source models trained end-to-end for long-horizon tasks and the platform their research team uses to build them. The company aims to provide AI companies, enterprises, and research teams with the ability to own their superintelligence by integrating their workflows, tools, data, and feedback loops. Prime Intellect trains open frontier models and deploys the same stack to customers, covering the full spectrum of training, deploying, and continuously improving models, including compute, large-scale RL, environments, sandboxes, evals, and deployment. The company has secured $150M in funding from prominent investors and individuals in the AI and infrastructure space. They are seeking individuals passionate about building at the intersection of frontier research, real infrastructure, and go-to-market strategies in a nascent category.

Requirements

Strong systems engineering experience in AI/ML infrastructure, especially around large-scale model training or inference.
Deep familiarity with PyTorch and distributed training frameworks such as PyTorch Distributed, DeepSpeed, FSDP, Megatron, vLLM, Ray, or related tooling.
Experience optimizing training performance across kernels, memory movement, communication overhead, or parallelization strategy.
Hands-on experience with large-scale training techniques including data parallelism, tensor parallelism, and pipeline parallelism.
Strong understanding of GPU architecture, profiling, and performance debugging.
Ability to identify bottlenecks across the stack and drive improvements from first principles.
Comfort working in a fast-moving environment with ambiguous problems and high ownership.

Nice To Haves

Experience writing or optimizing CUDA / Triton kernels.
Experience with compiler or runtime optimization for ML systems.
Experience working on RL training infrastructure, rollout systems, or asynchronous training pipelines.
Experience with multi-node GPU clusters and high-performance networking.
Contributions to open-source ML systems or infrastructure projects.
Interest in publishing technical work or sharing insights through engineering blogs and technical writing.

Responsibilities

Build and optimize the distributed training infrastructure for pre-training and large-scale RL training workloads by contributing to the prime-rl framework.
Improve end-to-end training efficiency across compute, memory, networking, and scheduling layers.
Design and implement low-level performance optimizations, including kernels, communication paths, and runtime improvements.
Work on distributed training systems spanning data, tensor, and pipeline parallel workloads.
Help shape the architecture of the RL training stack, including async rollout and post-training systems.
Contribute to open-source libraries and internal infrastructure used for frontier-scale model training.
Collaborate closely with researchers and infrastructure engineers to translate bottlenecks into concrete systems improvements.
Stay at the frontier of training systems, inference systems, compiler/runtime tooling, and hardware-aware optimization techniques.