Senior System Software Engineer - Dynamo-Triton Inference Server
About The Position
NVIDIA is hiring a Senior System Software Engineer for its GPU-accelerated deep learning software team, working on the Dynamo-Triton Inference Server. The team builds a high-performance AI inference platform to make designing and deploying AI models easier and more accessible. The role focuses on developing GPU-accelerated inference-serving software and contributing to the convergence of the Triton Inference Server and NVIDIA Dynamo stacks into a unified platform for both LLM and non-LLM workloads. The work balances production robustness, prediction throughput and latency, customer adoption, and the development and adoption of emerging inference technologies. The engineer will also participate in the open-source deep learning software community.
Requirements
- MS or PhD in Computer Science or a relevant field, or equivalent experience.
- 12+ years of professional experience working on deep learning software.
- Excellent Rust and C++ skills; familiarity with Python.
- Strong programming and software design skills, including debugging, performance analysis, and test design.
- Experience with high-scale distributed systems and ML systems.
- Strong communication skills and ability to work in a fast-paced, agile team environment.
Nice To Haves
- Prior experience with AI frameworks and engines such as TensorRT, PyTorch, ONNX, OpenVINO, vLLM, or TRT-LLM.
- Knowledge of GPU memory management, cache management, or high-performance networking.
- Experience with distributed systems programming.
- Experience contributing to a large open-source project, including GitHub, bug tracking, branching and merging code, open-source licensing issues, and handling patches.
Responsibilities
- Develop GPU-accelerated AI inference-serving software.
- Contribute to feature development and drive broad customer adoption.
- Drive convergence of the Triton Inference Server and NVIDIA Dynamo stacks to establish a unified, high-performance inference platform with feature parity for LLM and non-LLM workloads.
- Participate actively in the open-source deep learning software engineering community.
- Build robust software for production server or cloud environments.
- Optimize and balance prediction throughput and latency.
- Develop and adopt next-generation inference technologies.
Benefits
- Equity
- Benefits
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
