Head of Engineering

Inferact•San Francisco, CA

2d•Onsite

About The Position

Inferact's mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and faster. Founded by the creators and core maintainers of vLLM, we sit at the intersection of models and hardware, a position that took years to build. We're looking for a Head of Engineering to build and lead the organization developing the systems that power vLLM and Inferact. This role requires an engineering leader with genuine technical credibility at the inference layer—someone who understands GPU and accelerator performance, inference runtimes, ML systems optimization, and hardware-software co-design deeply enough to earn the trust of exceptional staff-level engineers. You'll partner closely with the founders to scale a senior-heavy, highly specialized engineering team while preserving the technical rigor, speed, and ownership that made vLLM successful. You'll recruit and develop rare ML systems talent, translate ambitious research and infrastructure work into a focused execution plan, strengthen how teams operate, and help Inferact deliver reliable, high-performance inference across models, hardware, and deployment environments.

Requirements

Bachelor's degree or equivalent experience in computer science, engineering, machine learning, systems, or a related field.
Engineering leadership experience building and scaling highly specialized teams in LLM inference, ML systems, GPU or accelerator software, distributed systems, or closely related infrastructure.
Deep technical credibility at the inference layer, including hands-on understanding of inference runtimes, GPU or accelerator optimization, kernels, memory and communication bottlenecks, and hardware-software tradeoffs.
Ability to distinguish core inference-engine work from the routing, orchestration, and application layers above it, with opinions grounded in direct technical experience.
A strong record of recruiting, assessing, and retaining senior engineers, staff-level ICs, PhDs, and research-adjacent engineers in a production engineering environment.
Experience translating technically ambitious work into clear priorities, accountable ownership, execution plans, and durable engineering operating mechanisms.
Ability to remain close enough to the work to identify risks, pattern-match on difficult technical problems, and unblock teams without becoming a bottleneck or displacing technical ownership.

Nice To Haves

Experience leading teams responsible for LLM serving, vLLM, SGLang, model execution, inference performance, GPU kernels, compiler or runtime systems, or distributed AI infrastructure.
Experience scaling a small, senior-heavy engineering organization where the relevant talent market is narrow and technical quality matters more than headcount growth.
Experience integrating research-oriented or PhD talent into production teams, including setting expectations, structuring work, and building effective collaboration with product-focused engineers.
Strong judgment across organizational design, hiring, performance management, technical planning, execution cadence, and cross-functional decision-making.
Built or led engineering teams working directly on GPU or accelerator-level inference performance, ML compilers, kernels, runtimes, or hardware-software co-design.
Contributed to or led teams around open-source ML systems projects such as vLLM, SGLang, PyTorch, Ray, Triton, XLA, ROCm, or related infrastructure.

Responsibilities

Build and lead the organization developing the systems that power vLLM and Inferact.
Scale a senior-heavy, highly specialized engineering team while preserving technical rigor, speed, and ownership.
Recruit and develop rare ML systems talent.
Translate ambitious research and infrastructure work into a focused execution plan.
Strengthen how teams operate.
Help Inferact deliver reliable, high-performance inference across models, hardware, and deployment environments.
Represent the engineering organization credibly with open-source contributors, hardware partners, cloud providers, customers, candidates, and investors.
Scale an engineering organization through an inflection point while preserving high technical standards, fast iteration, and direct ownership.
Recruit successfully from a global, highly competitive ML systems talent pool and build relationships with technical communities beyond traditional candidate pipelines.
Led engineering in an early-stage AI infrastructure, developer infrastructure, distributed systems, or open-source company.