Sr. Software Engineer, Training/Inference (C++)

SpaceXSI•Palo Alto, CA

13h•$220,000 - $357,000

About The Position

SpaceXSI is building a high-performance inference platform that serves Grok to millions of users every day. The Senior Software Engineer, Training/Inference (C++) will design and optimize large-scale model-serving systems end to end, spanning distributed infrastructure and low-level optimizations. The role’s work directly affects the speed and reliability of Grok interactions at massive scale. The position includes developing distributed serving infrastructure such as global KV cache, continuous batching, load balancing, and auto-scaling, as well as improving inference through GPU kernels, quantization, speculative decoding, and tail-latency optimization. It also supports research into test-time compute scaling, RL rollout, and model-hardware co-design.

Requirements

Bachelor’s degree in computer science, computer engineering, or a science discipline.
5+ years of low-level system programming experience in C/C++ or Rust.

Nice To Haves

Experience with large-scale, high-concurrency production serving.
Experience with GPU inference engines such as vLLM, SGLang, Triton, or TensorRT-LLM.
Strong background in system optimizations, including batching, caching, load balancing, and parallelism.
Low-level inference optimization experience, including GPU kernels and code generation.
Experience with algorithmic inference optimizations, including quantization, speculative decoding, distillation, and low-precision numerics.
Experience with testing, benchmarking, and reliability of inference services.
Experience designing and implementing CI/CD infrastructure for inference.

Responsibilities

Architect and implement scalable distributed infrastructure for model serving, including load balancing, auto-scaling, batch scheduling, and global KV cache.
Optimize inference latency and throughput under real production workloads.
Build and operate high-concurrency serving systems with excellent tail latency and high reliability at the scale required to serve millions of users.
Benchmark, fine-tune, and accelerate inference engines, including low-level GPU kernel work and code generation.
Develop tools to trace, replay, and fix issues across the stack, from orchestration to GPU kernels.
Create CI/CD infrastructure for endpoint deployment, image publishing, and inference engine updates.
Accelerate research on scaling test-time compute, RL rollout, and model-hardware co-design for next-generation systems.