Sr. Software Engineer, Training/Inference (C++)

SpaceXSI • Palo Alto, CA

1d • $200,000 - $375,000

About The Position

SpaceXSI is building a high-performance inference platform that serves Grok to millions of users. The Software Engineer, Training/Inference (C++) designs and optimizes large-scale model-serving systems end to end, spanning distributed infrastructure such as global KV cache, continuous batching, load balancing, and auto-scaling, as well as low-level optimizations including GPU kernels, quantization, speculative decoding, and tail latency. The role’s work directly affects the speed and reliability of Grok for users at massive scale.

Requirements

• Bachelor’s degree in computer science, computer engineering, or a science discipline.
• At least 5 years of low-level system programming experience in C/C++ or Rust.

Nice To Haves

• Experience with large-scale, high-concurrency production serving.
• Experience with GPU inference engines such as vLLM, SGLang, Triton, or TensorRT-LLM.
• Strong background in system optimizations, including batching, caching, load balancing, and parallelism.
• Experience with low-level inference optimizations, including GPU kernels and code generation.
• Experience with algorithmic inference optimizations, including quantization, speculative decoding, distillation, and low-precision numerics.
• Experience with testing, benchmarking, and reliability of inference services.
• Experience designing and implementing CI/CD infrastructure for inference.

Responsibilities

• Architect and implement scalable distributed infrastructure for model serving, including load balancing, auto-scaling, batch scheduling, and global KV cache.
• Optimize inference latency and throughput under real production workloads.
• Build and operate high-concurrency serving systems with excellent tail latency and high reliability at a scale sufficient to serve millions of users.
• Benchmark, fine-tune, and accelerate inference engines, including low-level GPU kernel work and code generation.
• Develop tools to trace, replay, and fix issues across the stack, from orchestration to GPU kernels.
• Create robust CI/CD infrastructure for endpoint deployment, image publishing, and inference engine updates.
• Accelerate research on scaling test-time compute, RL rollout, and model-hardware co-design for next-generation systems.

Benefits

• Equity.
• Comprehensive medical coverage.
• Vision coverage.
• Dental coverage.
• Access to a 401(k) retirement plan.
• Short- and long-term disability insurance.
• Life insurance.
• Various discounts and perks.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior