Principal Engineer, Rack-Scale GPU Architecture

DigitalOcean • Seattle, WA

5d • Hybrid

About The Position

DigitalOcean is seeking a Principal Engineer to lead the architecture, orchestration, and operation of rack-scale GPU systems for inference. This role is critical as the industry shifts from server-centric to rack-centric GPU capacity, with new hardware like GB300 NVL72 and future generations. The engineer will define the reference architecture for rack-scale inference, focusing on NVLink domain allocation, RDMA fabric design, Kubernetes topology awareness, and failure handling. This position involves collaboration with data center, networking, and inference platform teams, as well as direct engagement with NVIDIA for pre-silicon enablement and bringup. The goal is to enable DigitalOcean to serve trillion-parameter models with competitive economics.

Requirements

• 12+ years in large-scale compute, HPC, or GPU infrastructure engineering, with hands-on production systems responsibility.
• Deep understanding of GPU interconnect and system topology: NVLink, NVSwitch, NVLink domains, PCIe, NUMA, and their impact on workload placement.
• Strong RDMA networking background: RoCEv2 or InfiniBand at scale, GPUDirect RDMA, congestion control, lossless fabric tuning, and rail-optimized cluster topologies.
• Practical NCCL expertise: debugging, algorithm/protocol selection, topology detection, and diagnosing performance issues.
• Substantial Kubernetes depth for accelerated workloads: device plugins, DRA, scheduling extensions, operators, and understanding Kubernetes model limitations with rack-scale hardware.
• Strong systems programming and automation skills in Go and Python.
• Comfort at the driver, firmware, and BMC layer when necessary.
• Experience operating fleets with routine hardware failure, including diagnostics, automation, and blast-radius thinking.
• Excellent written and verbal communication skills.
• Track record of leading architecture across organizational boundaries (hardware, networking, platform, product).

Nice To Haves

• Direct experience bringing up GB200 or GB300 NVL72 systems, or comparable rack-scale platforms, in a production environment.
• Experience with multi-node inference for very large models (expert or pipeline parallelism spanning NVLink domains, placement constraints).
• Familiarity with liquid-cooled and high-density rack deployments, and their operational realities.
• Contributions to relevant open source projects (e.g., Kubernetes SIG-Node/scheduling, NVIDIA GPU/Network Operator, DRA drivers, Kueue, LeaderWorkerSet, NCCL).
• Experience running heterogeneous fleets across multiple accelerator vendors and generations under a single orchestration model.

Responsibilities

• Owning the end-to-end reference architecture for rack-scale inference systems (e.g., GB300 NVL72, Vera Rubin–class NVL144 and successors), from fabric design to customer scheduling.
• Defining NVLink domain partitioning, allocation, and isolation for tenants and workloads, including multi-node NVLink and IMEX domain provisioning, fabric manager configuration, and blast radius assessment.
• Architecting and tuning the scale-out RDMA fabric (RoCEv2 or InfiniBand) with rail-optimized topologies, congestion control, and GPUDirect RDMA paths, optimizing NCCL for performance.
• Making Kubernetes topology-aware for these systems, including DRA for GPU/NVLink-domain allocation, multi-node inference primitives (LeaderWorkerSet, gang scheduling), Topology Manager, NUMA alignment, and underlying GPU/Network Operators.
• Driving multi-node disaggregated serving in partnership with the inference platform team, mapping prefill/decode pools to NVLink domains and rail boundaries, and managing KV cache movement.
• Owning failure semantics and serviceability at rack scale, including health checking, DCGM-based diagnostics, drain/repair workflows, degraded-domain scheduling, and fleet-wide firmware/driver lifecycle management.
• Establishing validation and burn-in methodologies to qualify racks for production, including collective bandwidth/latency characterization, NCCL/end-to-end inference benchmarks, and acceptance gates.
• Partnering with data center engineering on power, liquid cooling, and rack density constraints to ensure deployability and cost-effectiveness.
• Working directly with NVIDIA and ODM partners on pre-silicon enablement, early hardware bringup, and roadmap feedback.
• Setting technical direction, mentoring senior and staff engineers, and representing DigitalOcean upstream, at conferences, and in customer architecture reviews.

Benefits

• Competitive salary range
• Bonus potential based on company and individual performance
• Equity compensation (grants upon hire, Employee Stock Purchase Program)
• Reimbursement for relevant conferences, training, and education
• Access to LinkedIn Learning
• Employee Assistance Program
• Local Employee Meetups
• Flexible time off policy
• Competitive array of benefits to support well-being (specifics may vary by location)

What This Job Offers

• Job Type: Full-time
• Career Level: Principal
• Education Level: No Education Listed