Engineering Lead, Inference Optimization

Venice.ai • Sheridan, WY

3d

About The Position

Venice is hiring an Engineering Lead to own its technical strategy for inference performance. The role combines hands-on individual-contributor work with management of a small team, and includes recruiting and leading Venice’s Inference Optimization Team. The lead will shape the company’s technical strategy and help build a team focused on delivering high inference performance at scale while supporting Venice’s privacy-focused AI platform.

Requirements

• At least 8 years of experience in performance optimization or HPC, with deep GPU architecture and parallel programming knowledge.
• At least 5 years of experience leading engineering teams.
• Proficiency in Python, Rust, or Go.
• Hands-on experience with at least one production LLM inference engine, such as vLLM or SGLang, running at high volume in production.
• Demonstrated experience with LLM inference optimization techniques, including continuous batching, PagedAttention/KV cache management, speculative decoding, quantization, CUDA graphs, and torch.compile.
• Fluency with quantization tradeoffs, both qualitative and quantitative.
• Experience with distributed inference strategies, including tensor parallelism, pipeline parallelism, and MoE parallelism, in multi-GPU and multi-node environments.
• Fluency with GPU profiling tools such as Nsight Systems, Nsight Compute, and PyTorch Profiler, and a bias toward measuring before optimizing.

Nice To Haves

• Hands-on kernel development experience.
• C++/CUDA proficiency.
• Experience optimizing diffusion or image-model inference.
• Experience with custom Triton kernels.
• Contributions to open-source inference frameworks.

Responsibilities

• Own Venice’s technical strategy for inference performance.
• Recruit and lead the Inference Optimization Team at Venice.
• Optimize Venice’s GPU infrastructure across architectures such as H200s and B300s.
• Improve latency, throughput, and cost per token for LLM inference workloads.
• Build reproducible benchmarking harnesses across inference engines such as vLLM and SGLang to identify the optimal engine, quantization scheme, and parallelism strategy for each workload and GPU SKU.
• Work with the inference routing system to optimize multivariate inference load-balancing algorithms.
• Evaluate emerging inference optimization techniques, including custom CUDA/Triton kernels, novel attention variants, new quantization schemes, and compilation stack improvements.
• Evaluate emerging inference hardware, including FPGAs, ASICs, and custom silicon, for viability in Venice’s stack.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed