Performance Engineer - Deep Learning
About The Position
NVIDIA’s Deep Learning models performance engineering team is hiring software engineers at multiple experience levels to build and optimize libraries and tools that help Deep Learning researchers and engineers design, develop, and deploy efficient AI applications. The team builds optimizations into the PyTorch and JAX open-source frameworks to improve performance across NVIDIA’s AI stack, collaborating with teams across NVIDIA and the broader open-source community. The role focuses on developing and supporting Transformer Engine, an open-source library for accelerating Large Language Model training; conducting systems research to improve Deep Learning performance; implementing, benchmarking, and optimizing models to scale on NVIDIA GPUs and systems; contributing to community benchmarks such as MLPerf; supporting open-source users, enterprise customers, and partners; and influencing future NVIDIA hardware and platform software.
Requirements
- Bachelor’s degree or equivalent experience in Computer Science, Electrical Engineering, or a related field.
- At least 2 years of experience programming in C++ and Python.
- Strong background, experience, or coursework in parallel systems programming, preferably on GPUs.
- Knowledge of computer architecture, code optimization, and/or operating systems.
- Proven experience developing large software projects.
- Excellent verbal and written communication skills.
Nice To Haves
- Experience with PyTorch, JAX, or another Deep Learning framework.
- Experience with performance analysis, profiling, and code optimization, especially for multi-GPU or multi-node systems.
- Knowledge of modern LLM architectures, attention mechanisms, and/or low-level Deep Learning libraries such as cuBLAS, cuDNN, and cuSOLVER.
- Experience writing GPU kernels using CUDA, OpenAI Triton, CuTeDSL, Pallas, or similar libraries.
- Past contributions to the open-source community and/or experience working with multidisciplinary teams.
Responsibilities
- Build and support Transformer Engine, the open-source library for accelerating the training of Large Language Models.
- Collaborate on systems research to improve Deep Learning model performance, including extremely low-precision training and parallelism methods.
- Implement, benchmark, and optimize new Deep Learning models from research to scale efficiently on NVIDIA GPUs and systems.
- Build and contribute to NVIDIA submissions to community benchmarks such as MLPerf.
- Engage with the open-source community and support enterprise customers and partners by delivering the benefits of NVIDIA’s latest hardware and software innovations.
- Influence the design of future hardware generations and core platform software components for NVIDIA hardware and systems.
Benefits
- Eligible for equity and benefits.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
