Research Scientist / Engineer – Training Infrastructure

Luma • Redwood City, CA

21d

About The Position

You'll build the distributed systems that train Luma's large-scale multimodal models across thousands of GPUs, so researchers can focus on innovation on top of reliable, efficient, scalable infrastructure. This is hard PyTorch, CUDA, and distributed-systems work — advanced parallelism, training stability, and utilization across massive clusters. It fits an engineer who's solved real problems training foundation models at scale. If you haven't worked at the level of FSDP and multi-node training, this is the wrong depth.

Requirements

• Extensive distributed PyTorch training and parallelisms in foundation-model training.
• Deep understanding of GPU clusters, networking, and storage systems.
• Familiarity with communication libraries (NCCL, MPI) and distributed-system optimization.

Nice To Haves

• Strong Linux systems administration and scripting.
• Experience managing training runs across 100+ GPUs.
• Experience with containerization, orchestration, and cloud infrastructure.

Responsibilities

• Design, implement, and optimize efficient distributed training systems for models across thousands of GPUs.
• Research and implement advanced parallelization (FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel).
• Build monitoring, visualization, and debugging tools for large-scale training runs.
• Optimize training stability, convergence, and resource utilization across massive clusters.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed