Senior Software Engineer, Distributed Systems - NIM Factory

NVIDIA•Santa Clara, CA

About The Position

NVIDIA is seeking a senior engineer to design and build factory infrastructure and automation for NVIDIA Inference Microservices (NIMs). The role focuses on developing an efficient, scalable, and reliable automation factory that turns AI models into deployable NIM services validated for performance and accuracy across cloud, on-premises, Kubernetes, and local NVIDIA GPU environments. The engineer will help build factory capabilities including infrastructure, pipelines, backends, Docker builds, test harnesses, metrics, performance engineering, and log ingestion. The position involves shaping technical strategies and roadmaps, designing interfaces and data models, improving schema design and observability, and developing scalable and reliable factory components. The engineer will work with technical leaders and AI model teams to understand requirements, improve productivity, define metrics, and drive improvements based on user feedback. The role also includes mentoring and collaborating across teams.

Requirements

Advanced programming skills and a history of building distributed and compute systems, backend services, microservices, and cloud technologies.
Experience working effectively with cross-functional teams, principals, and architects across organizational boundaries.
Experience mentoring and growing teams and team members, with flexibility to adjust direction and expectations to meet customer needs.
Deep technical expertise in distributed containerized applications using technologies such as Docker, Kubernetes, Cloud Endpoints, Helm, and Prometheus.
Passion for building rich microservice applications and build and test automation pipelines.
Excellent interpersonal skills and ability to lead cross-functional efforts.
Proven experience debugging and analyzing the performance of distributed microservices or cloud systems.
BS or MS in Computer Science, Computer Engineering, or a related field, or equivalent experience.
8+ years of experience developing performant microservices, cloud software, and/or tooling.

Nice To Haves

Experience delivering event-driven applications using services such as Temporal, Kafka, or Redis, and ability to discuss the trade-offs among these choices.
History of building and deploying containers for microservices in cloud and on-premises environments and their associated CI/CD pipelines.
Prior experience working in large-scale full-stack development.

Responsibilities

Develop a factory pipeline that accepts an AI model and produces a deployable service validated across cloud, on-premises, and Kubernetes environments.
Work with the team to define and deliver rapid iterations on technical strategies and roadmaps for the NIM factory.
Design interfaces, data models, and schemas, and expand observability across the factory pipeline and its compute infrastructure.
Design and develop scalable, reliable factory components with technical leaders.
Collaborate with AI model teams to understand their requirements and build infrastructure that improves team productivity.
Define metrics and drive improvements based on user feedback.
Mentor and collaborate with colleagues and other teams.