Senior Solutions Architect, AI Inference
About The Position
NVIDIA is seeking a Senior Solutions Architect to help address technical challenges in scaling AI inference and token generation on finite GPU resources. The role supports strategic inference service provider partners and NVIDIA Cloud Partners, helping them improve performance, value, and reliability across large-scale infrastructure. The architect will develop expertise across the inference stack, from routing and rack-scale disaggregation to kernel optimization. They will build and operate inference recipes, contribute reference architectures for large GPU clusters, and guide partners and customers—including executives, engineers, and datacenter developers—in scaling their systems and businesses. The position also involves developing technical approaches internally, sharing product insights, and helping the industry adopt effective inference practices.
Requirements
- At least 6 years in Solutions Architecture, Solution Engineering, Forward-Deployed Engineering, or a similar role, driving customer engagements to deploy distributed systems.
- At least 2 years working with AI workloads on Kubernetes.
- Experience with at least one of NVIDIA Dynamo, Triton Inference Server, or TensorRT-LLM for model optimization and serving.
- Deep knowledge of modern inference best practices, including disaggregated serving, multi-tier KV cache management, speculative decoding, quantization, and custom inference kernels.
- Hands-on experience designing full-stack agents with modern sandboxing, memory and retrieval systems with evaluation, skill design, and governance.
- Bachelor’s degree in Computer Science or Engineering, or equivalent experience.
Nice To Haves
- Prior experience deploying NVIDIA inference technologies such as Dynamo, NIXL, and Grove.
- Experience with model customization techniques such as SFT, DPO, GRPO, and RLVR, and corresponding data curation techniques.
- Contributions to relevant open-source projects.
Responsibilities
- Work with inference partners building advanced inference services, explain the value of NVIDIA’s stack, and bring product insights back to NVIDIA.
- Build and operate inference recipes using tools such as NVIDIA Dynamo, distributing tasks among GPU workers to improve efficiency.
- Accelerate inference pipelines using TensorRT-LLM, vLLM, SGLang, and other backends, integrating them with disaggregated inference.
- Promote DevOps best practices, including Kubernetes cluster management, compute fabric configuration, and observability.
- Provide mentorship and technical leadership to customers and internal teams.
- Guide customers through deploying disaggregated inference systems and resolve complex issues.
- Develop reference architectures and guide partners and the broader industry in adopting them.
Benefits
- Competitive salaries.
- Comprehensive benefits package.
- Equity.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
