AI Inference Framework Engineer

quadric•Burlingame, CA

17h•$160,000 - $195,000•Onsite

About The Position

Quadric is hiring an AI Inference Framework Engineer for its AI Application team. The role focuses on making Quadric’s General Purpose Neural Processing Unit (GPNPU) a first-class backend in established AI frameworks, including llama.cpp, PyTorch, vLLM, and Optimum, so customers can move applications from GPUs to Quadric with minimal changes. The engineer will own backend integration from memory management and dispatch through serving features, collaborating with kernel, runtime, and compiler teams. The position involves integrating the GPNPU backend into AI compute frameworks; implementing backend interfaces, memory allocation, compute dispatch, registration, operators, and kernels; and building model compute pipelines. It also covers serving capabilities such as KV-cache management, paged attention, continuous batching, and speculative decoding. The engineer will optimize performance and accuracy, validate results against reference backends, and support quantization formats including GGUF (Q4/Q8), GPTQ/AWQ, int8, and fp16. Within the first 6–12 months, success means bringing a new LLM/VLM family end to end onto the GPNPU backend with accuracy matching the reference, delivering measurable tokens-per-second or latency gains on a frontier model, and shipping a serving feature such as paged KV cache in a production release. The role is based at Quadric’s downtown Burlingame office. The position offers a base salary of $160,000–$195,000, equity, and a discretionary annual performance bonus, along with the listed US-based benefits. Quadric describes its culture as collaborative, low-ego, focused on ownership and impact, and oriented toward innovative disruption.

Requirements

At least 3 years of experience building or extending ML frameworks or inference runtimes.
BS/MS in computer science, computer engineering, or a related field.
Strong C/C++ and Python skills.
Solid understanding of framework internals, including tensor memory management, graph scheduling, and operator dispatch.
Solid understanding of AI compute performance analysis and optimization.
Fluency with AI coding agents such as Claude Code or Cursor, and interest in building agent-driven workflows for backend and model bring-up.

Nice To Haves

Experience writing an accelerator backend (such as CUDA, Metal, Vulkan, Hexagon, or another NPU) for an open-source framework.
Contributions to open-source frameworks such as PyTorch, llama.cpp, or vLLM.
Understanding of low-level AI compute pipelines involving CPU/NPU and DMA.
Understanding of graph compilers and kernel fusion.

Responsibilities

Integrate the GPNPU backend into AI compute frameworks such as llama.cpp, PyTorch, vLLM, and Optimum.
Implement framework backend interfaces, memory allocation, compute dispatch, registration, and operators or kernels.
Implement model compute pipelines, including prefill, decode, batching, multi-sequence, and multimodal processing.
Support serving features including KV-cache management, paged attention, continuous batching, and speculative decoding.
Optimize framework performance and accuracy for the GPNPU backend.
Reduce host-device dispatch overhead, implement graph capture and replay, integrate efficient kernels, and minimize latency.
Support quantization formats including GGUF (Q4/Q8), GPTQ/AWQ, int8, and fp16.
Validate accuracy against reference backends using perplexity and task benchmarks, and drive improvements.
Work closely with kernel, runtime, and compiler teams.
Bring up a new LLM/VLM family end to end on the GPNPU backend, achieve reference-matching accuracy, deliver measurable performance gains, and ship a serving feature in a production release.

Benefits

Medical, dental, and vision insurance from day one, with premiums covered at 99% for employees.
Company-paid life insurance.
Voluntary supplemental life insurance.
Short-term and long-term disability insurance.
Commuter support, including parking or Caltrain reimbursement.
Flexible spending account (FSA) and health savings account (HSA).
Equity.
Paid parental leave.
401(k) retirement plan.
Flexible PTO.
Winter holiday shutdown.
Catered lunch each day at the office.
Downtown Burlingame office close to shops, cafes, and local amenities.