Member of Technical Staff — Inference-TPU
About The Position
RadixArk is seeking a Member of Technical Staff to build high-performance inference and training systems using JAX, XLA, and Pallas. The role focuses on pushing model workloads to their limits on TPU hardware and developing SGLang-JAX and other infrastructure for efficient deployment of frontier models on Google's tensor processing units. The position involves optimizing end-to-end latency and throughput for large language model serving, designing distributed inference and training strategies, implementing custom Pallas kernels, and improving XLA compilation pipelines and HLO graph transformations. The successful candidate will collaborate with kernel engineers and compiler teams and contribute TPU optimization guides, benchmarks, and architectural insights to open-source projects.
Requirements
- At least 3 years of experience building production ML systems using JAX, Torch, XLA, or TPU-focused frameworks.
- Bachelor's or Master's degree in Computer Science, Electrical Engineering, or equivalent industry experience.
- Strong performance-tuning instincts across compiler and runtime layers.
- Experience with distributed inference systems such as SGLang or vLLM, or training frameworks such as Miles, Alpa, or Pathways.
- Proficiency in Python and demonstrated ability to write high-performance, production-quality code.
- Experience writing custom GPU, TPU, or AI accelerator kernels.
Nice To Haves
- Deep understanding of XLA internals, including HLO, MLIR, operator fusion, SPMD partitioning, and sharding strategies.
- Familiarity with Pallas for kernel development.
Responsibilities
- Build high-performance inference and training systems using JAX, XLA, and Pallas, including SGLang-JAX.
- Push large-model workloads to the limits of the newest TPU hardware.
- Optimize end-to-end latency and throughput for LLM serving on TPU infrastructure.
- Design and implement SPMD strategies for efficient distributed inference and training.
- Design and implement Pallas kernels for operations requiring customized low-level control and high performance.
- Profile and optimize XLA compilation pipelines and HLO graph transformations.
- Collaborate with kernel engineers and compiler teams to achieve performance improvements across the stack.
- Contribute TPU optimization guides, benchmarks, and architectural insights to open-source projects.
Benefits
- Equity
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
