Research Engineer - Inference

Prime Intellect • San Francisco, CA

1d • $150,000 - $350,000

About The Position

Prime Intellect is building an open superintelligence stack: infrastructure for frontier AI research made available to ambitious AI teams. Its Lab platform brings together compute, environments, evaluations, secure sandboxes, high-performance training, and deployment for post-training at frontier scale, including supervised fine-tuning, reinforcement learning, tool use, and agent workflows.

The Research Engineer - Inference will build inference systems that power Prime Intellect’s RL research, scaling toward 1–3T parameter models on NVL72 clusters. The work will improve rollout throughput and make frequent model updates practical. The role involves close collaboration with model researchers to balance model quality with inference efficiency and execution cost on target hardware.

Prime Intellect has raised $150M in total funding. The company offers flexible work arrangements, with the option to work remotely or in person at its San Francisco offices.

Requirements

• Experience building or optimizing GPU inference systems, with an understanding of inference-engine internals.
• Strong Python skills and experience with PyTorch or a comparable ML framework.
• Ability to profile end-to-end workloads and turn measurements into verified performance improvements.
• Experience debugging numerical or integration issues in ML systems.

Nice To Haves

• Contributions to vLLM or another inference engine.
• CUDA or Triton kernel work, particularly for low-precision inference.

Responsibilities

• Extend prime-rl’s vLLM integration to bring new model architectures into RL training, with small-scale validation before larger runs.
• Improve rollout throughput for long-horizon agents by tuning concurrency and KV-cache reuse as context lengths grow.
• Optimize multi-node MoE inference, including expert parallelism and the tradeoffs of separating prefill from decode.
• Reduce trainer-to-inference weight-update overhead through sharding-aware transfer and communication overlap, while keeping rollout workers on the intended policy version.
• Build on FP8 and evaluate lower precision; use trainer/sampler mismatch checks to catch numerical regressions and measure their effect on RL training.
• Work with model researchers on architecture choices that improve inference efficiency, balancing model quality against execution cost on target hardware.

Benefits

• Cash compensation range of $150,000–$350,000, including equity incentives.
• Flexible work arrangements, with the option to work remotely or in person at the San Francisco offices.
• Visa sponsorship and relocation assistance for international candidates.
• Quarterly team off-sites, hackathons, conferences, and learning opportunities.
• Opportunity to work with a talented, hard-working, and mission-driven team.

What This Job Offers

• Job Type: Full-time
• Education Level: No Education Listed