Inference Performance Engineer, Agent Driven Inference Optimization

NVIDIA•Santa Clara, CA

1d•$124,000 - $241,500

About The Position

NVIDIA is hiring a Senior Inference Performance Engineer to improve performance on large-scale AI inference benchmarks through an autonomous optimization framework. The role involves creating reusable optimization skills, workflows, and evidence-backed methodologies that AI agents can execute, reviewing and validating agent-generated experiments, and maintaining best-known configurations. The engineer will improve AI inference workloads to increase throughput per GPU and user interactivity. This includes exploring configuration options, parallelism, batching, KV cache handling, quantization, and speculative decoding. The role measures and optimizes aggregated and disaggregated serving architectures across TensorRT-LLM, SGLang, vLLM, and Dynamo on NVIDIA GPU platforms. It also uses profiling and analysis to identify performance headroom, drives fixes through measurement, and contributes upstream framework patches, optimized kernels, and deployment recipes while preserving model correctness. The engineer collaborates with serving framework, kernel, benchmarking, and GPU architecture teams to turn profiling insights into performance improvements. The position offers a competitive salary, equity, and benefits. Applications will be accepted at least until November 13, 2026. This posting is for an existing vacancy.

Requirements

Bachelor's, master's, or PhD in Computer Science, Computer Engineering, Electrical Engineering, Applied Math, or a related field, or equivalent experience.
At least 3 years of relevant engineering experience.
Extensive knowledge of AI model execution efficiency and optimization, including continuous batching, throughput-latency tradeoffs, KV cache and memory limitations, parallel processing techniques, MoE serving, quantization, and serving SLAs.
Hands-on GPU workload benchmarking and profiling experience using tools such as Nsight Systems, Nsight Compute, CUPTI, or PyTorch profiler, including interpreting kernel-level performance data.
Strong Python engineering skills and ability to navigate and modify large C++/CUDA serving codebases.
Rigorous experimental methodology, including controlled single-variable comparisons, reproducible benchmarks, and evidence-backed optimization decisions.
Strong written and verbal communication skills, including the ability to explain performance tradeoffs clearly to people and document them for autonomous systems.

Nice To Haves

Direct contributions to TensorRT-LLM, vLLM, SGLang, FlashInfer, Dynamo, or comparable inference frameworks.
Experience with disaggregated serving, wide expert-parallel MoE inference, KV cache transfer, or NCCL/NIXL/NVSHMEM communication at multi-node scale.
CUDA kernel authorship or optimization experience on Hopper or Blackwell architectures, focusing on Tensor Cores, TMA, and warp specialization.
Proven results on public inference benchmarks such as MLPerf Inference or SemiAnalysis InferenceX.
Experience building or operating agentic AI workflows to automate engineering tasks.

Responsibilities

Develop reusable optimization skills, workflows, and evidence-backed methodologies for AI agents to perform autonomously.
Review agent-generated experiments, validate findings, and curate best-known configurations.
Improve AI inference workload throughput per GPU and user interactivity by exploring configuration options, parallelism, batching, KV cache handling, quantization, and speculative decoding.
Measure and optimize aggregated and disaggregated serving architectures across TensorRT-LLM, SGLang, vLLM, and Dynamo on NVIDIA GPU platforms.
Profile workloads with Nsight Systems, kernel traces, and internal analysis tools; use roofline and speed-of-light analysis to identify performance headroom.
Drive performance fixes from hypothesis through measurement.
Contribute serving framework patches, optimized kernels, and deployment recipes upstream while maintaining strict model correctness.
Collaborate with serving framework, kernel, benchmarking, and GPU architecture teams to deliver performance improvements.