Sr. Software Engineer, Training/Inference (C++)

xAI•Seattle, WA

About The Position

SpaceXSI is building a high-performance inference platform that serves Grok to millions of users. The Senior Software Engineer, Training/Inference (C++) will design and optimize large-scale model-serving systems end to end. The role spans distributed infrastructure—including global KV cache, continuous batching, load balancing, and auto-scaling—and low-level optimizations such as GPU kernels, quantization, speculative decoding, and tail-latency improvements. The work directly affects the speed and reliability of Grok interactions at scale. The engineer will also develop tools and deployment infrastructure, and support research into scaling test-time compute, RL rollout, and model-hardware co-design for next-generation systems.

Requirements

Bachelor’s degree in computer science, computer engineering, or a science discipline.
5+ years of low-level system programming experience in C/C++ or Rust.

Nice To Haves

Experience with large-scale, high-concurrency production serving.
Experience with GPU inference engines such as vLLM, SGLang, Triton, or TensorRT-LLM.
Strong background in system optimizations such as batching, caching, load balancing, and parallelism.
Low-level inference optimization experience, including GPU kernels and code generation.
Algorithmic inference optimization experience, including quantization, speculative decoding, distillation, and low-precision numerics.
Experience testing, benchmarking, and ensuring the reliability of inference services.
Experience designing and implementing CI/CD infrastructure for inference.

Responsibilities

Architect and implement scalable distributed infrastructure for model serving, including load balancing, auto-scaling, batch scheduling, and global KV cache.
Optimize inference latency and throughput under real production workloads.
Build and operate high-concurrency serving systems with excellent tail latency and high reliability at the scale required to serve millions of users.
Benchmark, fine-tune, and accelerate inference engines, including low-level GPU kernel work and code generation.
Develop tools to trace, replay, and fix issues across the stack, from orchestration to GPU kernels.
Create CI/CD infrastructure for endpoint deployment, image publishing, and inference engine updates.
Accelerate research on scaling test-time compute, RL rollout, and model-hardware co-design for next-generation systems.