Software Architect, NIM Factory

NVIDIA • Santa Clara, CA

101d • $272,000 - $425,500

About The Position

NVIDIA is the platform upon which every new AI-powered application is built. We are seeking a Software Architect to define and own the technical vision for the NVIDIA Inference Microservices (NIM) Factory. You will set the architectural direction for how we build, deploy, and scale enterprise-grade AI services to delight customers, while staying hands-on to guide our most critical implementations. The scope spans day-0 launches and the follow-through to harden them into enterprise-grade software, ensuring reliability, performance, and security across thousands of GPUs. You will shape our strategy for emerging challenges like disaggregated LLM inference and safeguard the long-term technical health of the platform.

Requirements

• 15+ years of experience building large-scale, production distributed systems.
• Consistent track record in a technical leadership or architect role, setting technical direction, and implementing.
• Deep architectural expertise in cloud-native technologies, including Kubernetes, containers, and microservices.
• Exceptional ability to mentor, and grow senior engineers with a passion for raising the technical bar of the entire organization.
• Proficiency in languages like Python for building tooling and services.
• Experience architecting solutions for GPU-accelerated or other high-performance computing workloads.
• Excellent communication and collaboration skills, with the ability to articulate complex technical concepts to diverse audiences and drive consensus.
• A degree in Computer Science, Computer Engineering, or a related field (BS or MS) or equivalent experience.

Nice To Haves

• Hands-on with LLM inference stacks (Triton Inference Server, TensorRT-LLM, vLLM).
• Experience optimizing large-model serving (KV cache sharding/paging, tensor/sequence parallelism, speculative decoding, dynamic batching).
• Experience architecting next-generation container build systems or CI/CD platforms at scale.
• Background with workflow orchestration engines (e.g., Temporal, Airflow) for complex, distributed processes.
• Expertise in designing multi-tenant, multi-cluster, or edge/air-gapped deployment architectures.

Responsibilities

• Define the end-to-end technical architecture for the NIM Factory, from container build systems and CI/CD to Kubernetes deployment patterns and runtime optimization.
• Drive technical strategy and roadmap, making high-impact decisions on frameworks, technologies, and standards that empower dozens of engineering teams.
• Architect and influence the design of workflow orchestration systems that underpin the NIM factory.
• Guide and support senior engineers throughout the organization in building a culture centered on technical excellence and innovation.
• Advocate for guidelines in software development, encompassing API composition, automation, observability, and secure supply chain management.
• Collaborate with leadership across research, backend, SRE, and product to align technical vision with product goals and influence technical roadmaps.

Benefits

• Competitive salaries
• Generous benefits package
• Equity eligibility

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: Bachelor's degree
• Number of Employees: 5,001-10,000 employees