ML Engineer, Inference Optimization

Build AI•San Francisco, CA

13h•Onsite

About The Position

Build AI is hiring an ML Engineer, Inference Optimization to reduce the cost and improve the speed of model inference. Since inference accounts for about 90% of compute spend, the role focuses on making inference economical enough to scale the company’s data engine and products. The engineer will work across research and product, helping ensure models that meet quality needs are also affordable to operate at scale. Build AI is a vertically integrated Physical AI data company spanning hardware, manufacturing, logistics, data collection, and model training. The team is fully in person in San Francisco (Financial District) and Shenzhen (Nanshan). Engineering and research are not separated: technical staff are expected to contribute to both and work across disciplines as needed. The company values engineering skills and describes its approach as learning from humans, following the “Bitter Lesson.”

Requirements

Strong ML or systems engineering background with hands-on inference optimization experience in areas such as serving, compilers, CUDA or kernels, or quantization.
Proficiency in Python and in C++ or Rust for performance-critical paths.
A cost-conscious approach that considers dollars and tokens or frames per second, not only accuracy metrics.
Familiarity with PyTorch or JAX and profiling tools.
Comfort working in a small research team and shipping under cost pressure.

Nice To Haves

Production experience with CUDA, kernels, compilers such as TVM, MLIR, or TensorRT, or quantization.
Experience owning GPU or accelerator cost as a first-class metric.
Experience with serving stacks for video or large models.
Understanding of memory hierarchy, data movement, and low-precision compute.

Responsibilities

Own inference performance, including latency, throughput, and cost per unit of work such as tokens, frames, or jobs.
Reduce compute costs through work on kernels, batching, quantization, compilation, serving, and hardware utilization.
Profile pipelines using tools such as Nsight and PyTorch Profiler, identify bottlenecks, and implement fixes.
Collaborate with research and product teams so models are accurate and affordable to run at scale.
Build serving and evaluation paths that make inference costs visible during experiments.
Measure cost as a first-class metric rather than considering it only after model quality is complete.

Benefits

Competitive pay.
Medical, dental, and vision packages with generous premium coverage.
$500 per month credit for waiving medical benefits.
Housing subsidy of $2,000 per month for employees living within walking distance of the office.
Relocation support for moves to San Francisco (Financial District) or Shenzhen (Nanshan).
Wellness benefits covering fitness, mental health, and more.
Daily lunch and dinner in the office.
Unlimited compute budget, subject to ROI justification.
Unlimited Codex and Claude credits.
Travel.