Senior Software Engineer, GPU Code Generation for Machine Learning Inference

NVIDIA•Us, CA

22h•Hybrid

About The Position

NVIDIA is seeking a Senior Software Engineer to join the TensorRT team and advance high-performance AI inference on NVIDIA GPUs. The team develops software to accelerate generative AI models for local and enterprise deployments, addressing GPU code generation, just-in-time compilation, and inference optimization across diverse models and hardware configurations. In this role, you will develop advanced GPU inference software for TensorRT, applying GPU optimization and compiler development expertise to generate efficient code and improve generative AI inference performance. You will work with engineers across GPU architecture, compilers, and AI software.

Requirements

Bachelor’s, master’s, or PhD in Computer Science, Computer Engineering, or a related field, or equivalent experience.
At least 4 years of relevant software development experience, including contributions to a substantial codebase or project.
Strong proficiency in C++ and experience building performance-critical software.
Understanding of GPU architecture and parallel programming concepts, including memory hierarchies, thread execution, synchronization, and resource utilization.
A rigorous approach to performance analysis and debugging, including the ability to validate numerical correctness and explain performance tradeoffs.
Strong problem-solving and communication skills, and the ability to work independently and collaborate across teams.

Nice To Haves

Experience developing or optimizing GPU kernels using CUDA, Triton, CuTe DSL, CUTLASS, or other GPU programming languages.
Experience with MLIR compiler infrastructure, including familiarity with intermediate representations, lowering, optimization passes, and GPU code generation.
Familiarity with NVIDIA Tensor Core architecture, mixed-precision computation, quantization, or low-level GPU performance analysis using tools such as NVIDIA Nsight.
Experience developing inference frameworks such as PyTorch, JAX, ONNX Runtime, TensorRT, TensorRT-LLM, vLLM, and SGLang.
A track record of measurable performance improvements in production software or contributions to GPU programming and compiler projects.

Responsibilities

Design and implement efficient GPU code generation techniques in TensorRT, balancing generated-code performance, compilation overhead, and startup latency.
Develop compiler optimizations, including operator fusion, dynamic scheduling, and specialization for model shapes, data types, and GPU architectures.
Profile workloads and analyze generated GPU code to identify bottlenecks and guide compiler and runtime improvements.
Develop robust, maintainable software in C++ and Python for tooling, testing, and performance analysis.
Collaborate with GPU architects, kernel developers, compiler engineers, and deep learning experts to guide hardware and software design.