Principal Software Engineer, GPU Compute
About The Position
As a Principal Software Engineer on Roblox's Compute team, you will be the technical anchor for the company's GPU and AI accelerator capabilities. The role focuses on the machine management layer and above: preparing GPU hosts for production, maintaining their health, and providing reliable compute for dependent workloads. You will own fleet-scale challenges involving driver and firmware management, GPU health, reliability, and performance across Roblox data centers and cloud environments. You will set technical direction for GPU compute and strengthen GPU expertise across the organization.
Requirements
- 10+ years of experience building and operating large-scale distributed systems and infrastructure.
- Deep, hands-on GPU expertise at the machine management layer and above, including GPU host provisioning, driver and firmware lifecycle, GPU health and reliability, and production accelerator operations.
- A track record as a compute expert, beyond fleet management, with experience scaling GPU or accelerator infrastructure that other teams depend on.
- Strong proficiency in Go or other well-structured programming languages.
- Experience operating GPU and AI workloads in production, including familiarity with CUDA, GPU scheduling, and high-performance networking such as NVLink, InfiniBand, and RoCE.
- A history of serving as the anchor expert for an organization's hardest GPU and compute problems, with the leadership to up-level other engineers.
Nice To Haves
- Familiarity with Kubernetes for GPU workloads.
- Familiarity with bare-metal concepts, including firmware, BMC/IPMI/Redfish, and OS imaging.
Responsibilities
- Serve as the GPU technical leader for the Compute team and partner across Kubernetes, Machine Bootstrap, Networking, and Cloud to drive GPU strategy end to end.
- Own the GPU host lifecycle above raw fleet management, including driver, firmware, and CUDA stack management, GPU health and telemetry, and remediation of GPU-specific failures such as XID errors, ECC, thermal issues, and NVLink and fabric faults.
- Architect how GPU capacity is exposed to compute platforms, including scheduling, isolation, and Kubernetes integration for GPU and AI workloads.
- Drive GPU reliability and performance at fleet scale by defining detection, diagnosis, and automated repair of unhealthy accelerators before they affect production.
- Evaluate and onboard new GPU and AI accelerator platforms, networking topologies such as NVLink, InfiniBand, and RoCE, and multi-node training and inference patterns.
- Establish standards, tooling, and APIs that enable other engineering teams to consume GPU compute safely and efficiently, reducing toil and raising the bar for the organization.
- Set technical direction for GPU compute and up-level the organization's GPU expertise.
Benefits
- Equity compensation for full-time employees.
- Benefits as described on the company's benefits page.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Principal
Education Level
No Education Listed
