Senior Performance Engineer - Deep Learning

NVIDIA•Santa Clara, CA

2d•$152,000 - $287,500

About The Position

NVIDIA’s Deep Learning models performance engineering team is hiring software engineers at all experience levels to build and optimize libraries and tools that help Deep Learning Researchers and Engineers design, develop, and deploy efficient AI applications. The team builds optimizations into mainstream open-source Deep Learning frameworks, including PyTorch and JAX, to improve performance across NVIDIA’s AI stack. It collaborates across NVIDIA and with the broader open-source community to deliver high-performance Deep Learning on NVIDIA platforms. The role focuses on developing and supporting Transformer Engine, an open-source library for accelerating Large Language Model training; conducting systems research to improve Deep Learning performance; and implementing, benchmarking, and optimizing emerging Deep Learning models for efficient scaling on NVIDIA GPUs and systems. The engineer will also contribute to community benchmarks, engage with open-source communities and enterprise customers, and influence future NVIDIA hardware and platform software designs.

Requirements

Bachelor’s degree or equivalent experience in Computer Science, Electrical Engineering, or a related field.
At least 3 years of experience in C++ and Python programming.
Strong background, experience, or coursework in parallel systems programming, preferably on GPUs.
Knowledge of computer architecture, code optimization, and/or operating systems.
Proven experience developing large software projects.
Excellent verbal and written communication skills.

Nice To Haves

Experience with PyTorch, JAX, or another Deep Learning framework.
Experience with performance analysis, profiling, and code optimization techniques, especially with multi-GPU or multi-node systems.
Knowledge of modern LLM architectures, attention mechanisms, and/or low-level Deep Learning libraries such as cuBLAS, cuDNN, and cuSOLVER.
Experience writing GPU kernels using CUDA, OpenAI Triton, CuTeDSL, Pallas, or similar libraries.
Past contributions to the open-source community and/or experience working with multidisciplinary teams.

Responsibilities

Build and support Transformer Engine, the open-source library for accelerating the training of Large Language Models.
Collaborate on systems research that improves Deep Learning model performance, including extremely low-precision training and parallelism methods.
Implement, benchmark, and optimize new Deep Learning models, such as LLMs, based on emerging research to scale efficiently on NVIDIA GPUs and systems.
Build and contribute to NVIDIA submissions for community benchmarks such as MLPerf.
Engage with the open-source community and support enterprise customers and partners by delivering the benefits of NVIDIA’s latest hardware and software innovations.
Influence the design of new hardware generations and core platform software components for NVIDIA hardware and systems.