Member of Technical Staff (Software Engineer, GPU Cluster Infrastructure)
About The Position
Perplexity is hiring a Member of Technical Staff to take ownership of infrastructure for its large GPU fleet, which spans multiple cloud providers and supports real-time AI inference. The role will build a unified, self-serve platform that lets inference engineers and researchers run training and inference workloads without managing GPU provisioning, cluster configuration, or provider-specific infrastructure. The position combines GPU fleet operations, Kubernetes orchestration, scheduling and placement across providers, and reliability engineering. It supports both long-running distributed training jobs and production inference services with high availability and low latency. The engineer will also partner with inference and cloud infrastructure teams to shape the platform architecture and roadmap.
Requirements
- Deep Kubernetes experience, including custom operators, CRDs, and multi-cluster federation.
- Experience managing GPU clusters at scale, including NVIDIA hardware, CUDA, and high-speed networking such as InfiniBand or RoCE.
- Experience orchestrating compute across multiple clouds, such as CoreWeave, AWS, or GCP, and understanding their differences.
- Strong distributed systems fundamentals, including scheduling, resource allocation, and fault tolerance under load.
- Ability to write infrastructure and systems-level code in Go, Rust, or C++.
- Experience supporting both long-running training jobs and high-availability inference services.
- Ability to own problems end-to-end and work effectively when the path forward is not laid out.
Nice To Haves
- Experience with inference serving stacks such as vLLM, SGLang, or TensorRT-LLM.
- Experience with Slurm or other HPC schedulers.
- GPU kernel experience in CUDA or Triton.
- Production experience with high-speed interconnects such as InfiniBand, RoCE, or RDMA.
- Experience with observability for ML workloads, such as Prometheus, Grafana, or Weights & Biases.
Responsibilities
- Design and own a self-serve compute platform for launching training jobs and operating inference services without requiring users to manage GPU provisioning, cluster configuration, or provider-specific infrastructure.
- Own GPU fleet provisioning, lifecycle management, reliability, and capacity integration across cloud providers.
- Build scheduling and placement logic that finds available capacity across providers, packs it efficiently, and assigns workloads to appropriate hardware.
- Support long-running distributed training jobs while maintaining the availability and latency of production inference services on the same fleet.
- Develop Kubernetes operators and CRDs and manage GPU orchestration across multiple clusters and providers.
- Build fault tolerance, autoscaling, and observability so workloads can withstand node loss, provider issues, and capacity changes without human intervention.
- Partner with inference and cloud infrastructure engineers to translate operational constraints into platform architecture and a roadmap.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
