Research Engineer, Infrastructure, Inference

Thinking Machines Lab•San Francisco, CA

20h•$400,000 - $500,000•Onsite

About The Position

Thinking Machines is hiring an infrastructure research engineer to design, optimize, and scale systems powering large AI models. The role focuses on making inference faster, more cost-effective, reliable, and reproducible so teams can advance model capabilities without infrastructure bottlenecks. The work supports performant and efficient inference for real-world applications and research, and ensures experiments, evaluations, and deployments run smoothly at scale. This is an evergreen posting for ongoing expressions of interest. Applications are reviewed continuously, but an immediate matching opportunity may not be available. Applicants are encouraged to reapply after gaining more experience, but not more than once every six months. Separate postings may also be available for specific projects or teams. The position is based in San Francisco, California. The expected annual salary range is $400,000–$500,000 USD, depending on background, skills, and experience. The company sponsors visas and offers relocation support as needed.

Requirements

Bachelor’s degree or equivalent experience in computer science, engineering, or a similar field.
Understanding of deep learning frameworks such as PyTorch or JAX and their underlying system architectures.
Experience with inference-serving systems optimized for throughput and latency, such as SGLang or vLLM.
Ability to thrive in a highly collaborative environment involving cross-functional partners and subject matter experts.
Bias for action and initiative to work across stacks and teams to help ensure work ships.
Strong engineering skills, including the ability to contribute performant, maintainable code and debug complex codebases.

Nice To Haves

Experience training or supporting large-scale language models with hundreds of billions of parameters or more.
Understanding of distributed compute systems, GPU parallelism, and hardware-aware optimizations.
Contributions to open-source ML or systems infrastructure projects such as SGLang, vLLM, PyTorch, Triton, DeepSpeed, or XLA.
Track record of improving research productivity through infrastructure design or process improvements.

Responsibilities

Work with researchers and engineers to bring cutting-edge AI models into production.
Collaborate with research teams to enable high-performance inference for novel architectures.
Design and implement techniques, tools, and architectures that improve performance, latency, throughput, and efficiency.
Optimize the codebase and compute fleet, including GPUs, to fully utilize hardware FLOPs, bandwidth, and memory.
Extend orchestration frameworks such as Kubernetes, Ray, and SLURM for distributed inference, evaluation, and large-batch serving.
Establish reliability, observability, and reproducibility standards across the inference stack.
Share learnings through internal documentation, open-source libraries, or technical reports that advance scalable AI infrastructure.