Senior Software Engineer, Distributed Systems - NIM Factory

NVIDIA•Santa Clara, CA

5d•$184,000 - $287,500

About The Position

NVIDIA is seeking a senior engineer to design and build factory infrastructure and automation for NVIDIA Inference Microservices (NIMs). The role focuses on creating efficient, scalable, and reliable systems that turn AI models into NIMs validated for performance and accuracy across cloud, customer-hosted, and local NVIDIA GPU deployments. The engineer will help build factory capabilities, including infrastructure, pipelines, backends, Docker builds, test harnesses, metrics, performance engineering, and log ingestion. The engineer will help shape the technical strategy and roadmap for the NIM factory, build and improve its infrastructure, and accelerate delivery of AI models on NVIDIA GPUs across heterogeneous hardware and software environments. The role involves collaboration with technical leaders and AI model teams to understand requirements and develop productive, reliable factory components.

Requirements

Advanced programming skills and a history of building distributed and compute systems, backend services, microservices, and cloud technologies.
Effective experience working with multifunctional teams, principals, and architects across organizational boundaries.
Experience mentoring and growing teams and team members, and flexibility to adjust direction and expectations based on customer needs.
Deep technical expertise in distributed containerized applications using technologies such as Docker, Kubernetes, Cloud Endpoints, Helm, and Prometheus.
Passion for building rich microservice applications and build and test automation pipelines.
Excellent interpersonal skills and ability to lead multifunctional efforts.
Proven experience debugging and analyzing the performance of distributed microservices or cloud systems.
BS or MS in Computer Science, Computer Engineering, or a related field, or equivalent experience.
8+ years of experience developing performant microservice or cloud software and/or tooling.

Nice To Haves

Experience delivering event-driven applications using services such as Temporal, Kafka, or Redis, and ability to discuss the pros and cons of these choices.
History of building and deploying containers for microservices in cloud and on-premises deployments, and their associated CI/CD pipelines.
Prior experience working with large-scale full-stack development.

Responsibilities

Develop a factory pipeline that takes an AI model and produces a deployable service validated across cloud, on-premises, and Kubernetes environments.
Define and deliver iterations on the group's technical strategies and roadmaps for the NIM factory.
Design interfaces, data models, and schemas.
Expand observability across the factory pipeline and its compute infrastructure.
Design and develop scalable and reliable factory components with technical leaders.
Collaborate with AI model teams to understand requirements and build infrastructure that improves their productivity.
Define metrics and drive improvements based on user feedback.
Mentor and collaborate with colleagues and other teams.