Senior Systems Engineer - AI Infrastructure

Clockwork.io•Palo Alto, CA

About The Position

Clockwork Systems is building infrastructure for fault-tolerant, high-performance distributed GPU training. The role involves working at the intersection of GPU systems, high-speed networking, and distributed coordination—designing and implementing systems that run at scale. This is a systems building role where the engineer will dig into internals, understand system failures under pressure, and design solutions that handle the complexities of distributed systems.

Requirements

Systems building experience: designed and built complex systems (e.g., Kernel subsystems, device drivers, OS-level components, distributed storage, databases, coordination systems, runtimes, profilers, performance tooling, network stacks, protocols, high-performance I/O systems, large-scale infrastructure at the systems layer)
Strong C/C++ in systems contexts
Deep understanding of concurrency, memory models
Experience reasoning about distributed system behavior: consistency, ordering, partial failures
Comfortable reading and modifying large, unfamiliar codebases

Nice To Haves

GPU programming (CUDA) or GPU systems experience
High-performance networking (RDMA, InfiniBand)
ML framework or runtime internals
Cluster scheduling or orchestration systems

Responsibilities

Design and implement low-level systems software for GPU clusters
Work with internals of frameworks like PyTorch, NCCL, CUDA runtime—not as a user, but modifying and extending them
Build components that make large-scale GPU training more reliable and efficient
Debug complex distributed/concurrent systems where failures are subtle and non-deterministic
Own systems end-to-end: from design through production