Head of Infrastructure

General Compute • San Francisco, CA

2d

About The Position

General Compute is a neocloud for alternative chips, providing infrastructure for AI inference workloads. The company purchases accelerator hardware, secures data-center capacity, and operates the hardware for customers. Its customers include frontier labs, fast-growing AI application companies, and asset-light clouds.

The Head of Infrastructure will own the inference-cloud infrastructure layer end to end. Initially, this includes the control plane, the gateway in front of the ASIC fleet, and the observability stack. Over the following 6–8 months, the infrastructure is expected to expand to a heterogeneous fleet of ASICs for decode and GPUs for pre-fill, along with ownership of the physical layer.

The first six months are hands-on, including Kubernetes manifests, dashboards, on-call work, and direct technical coordination with the ASIC partner. The role will build the on-call and incident-response practice from the ground up and hire and grow the infrastructure team.

Requirements

• At least 7 years of experience in infrastructure, SRE, or platform engineering, including some experience at a serious inference, ML, or HPC organization.
• Hands-on experience with Kubernetes at production scale, including debugging complex issues.
• Strong instincts for tail latency and the ability to consider p99 latency and utilization together.
• Comfort owning a vendor relationship where vendor bugs affect production.
• A track record of building observability practices that catch problems, not merely produce dashboards.
• Experience participating in on-call rotations through real incidents and the ability to discuss lessons learned.
• Interest in being the first infrastructure hire at an early-stage company.

Nice To Haves

• Experience operating non-NVIDIA accelerators in production, such as TPUs, ASICs, or alternative GPU vendors.
• Background with model-serving stacks such as vLLM, TGI, TensorRT-LLM, or SGLang.
• Data-center-scale network fabric experience, including RoCE or InfiniBand.
• Experience hiring and managing an infrastructure team.
• Comfort working at the hardware boundary, including firmware, drivers, and thermals.

Responsibilities

• Own and understand the inference control plane, modifying and extending the ASIC partner-provided configuration and eventually replacing parts of it.
• Own the gateway and load balancer in front of the fleet, including model placement, request routing, and tail-latency engineering based on live utilization and per-model SLOs.
• Own end-to-end observability, including per-request tracing from OpenRouter ingress to the accelerator, p50/p95/p99 dashboards, SLOs, and alerting.
• Plan capacity against the distributed traffic mix across the open-weight models served.
• Manage the operational side of the ASIC partnership and act as the technical point of contact for production issues routed through the partner's engineering team.
• Bring up the pre-fill side of the disaggregated architecture on a second hardware platform as it comes online.
• Build the on-call and incident-response practice from the ground up.
• Hire and grow the infrastructure team.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed