Senior Software Engineer, AI Frameworks

NVIDIA•Santa Clara, CA

1d•$152,000 - $287,500

About The Position

NVIDIA is seeking a Senior Software Engineer to integrate the NVIDIA Grove project within Dynamo and across leading open-source AI frameworks. The role develops production-grade software that enables Grove capabilities to be adopted, scaled, and operated across environments such as Dynamo, llm-d, Ray, PyTorch, and other emerging AI frameworks. The engineer will collaborate with engineering teams and the open-source community to deliver robust integrations, reference implementations, and developer-focused tooling.

Requirements

Bachelor’s, master’s, or Ph.D. in Computer Science, Electrical Engineering, or a related field, or equivalent experience.
At least 5 years of proven experience in a related field.
Hands-on experience integrating with at least one major AI framework or runtime, such as PyTorch, Ray, the Triton Inference Server ecosystem, distributed runtimes, or model-serving stacks.
Solid understanding of AI workloads, including model development basics, training-versus-inference tradeoffs, and performance considerations such as throughput, latency, batching, and memory.
Experience with distributed systems concepts, including RPC, scheduling, fault tolerance, and resource management.
Practical Kubernetes experience deploying and operating services or jobs, using Helm or Kustomize, and debugging clusters.
Familiarity with containers and cloud-native tooling, including Docker, container registries, and CI/CD pipelines.
Strong software engineering experience in Go, C++, and/or Python, with a track record of shipping reliable systems.
Strong interpersonal skills and ability to collaborate across teams and with open-source communities.
Exceptional collaboration, communication, and documentation habits.

Nice To Haves

Open-source contributions to Dynamo, PyTorch, Ray, llm-d, the Kubernetes ecosystem, or related ML infrastructure projects.
Experience with large-scale model serving, distributed inference, or multi-tenant AI platforms.
Experience building SDKs, APIs, or developer tooling that improves integration usability.
Knowledge of GPU performance profiling and optimization using Nsight tools or similar, and/or kernel-level performance tuning.
Experience with reproducibility, packaging, versioning, and compatibility testing across fast-moving dependencies.
Experience with Kubernetes operators or controllers.

Responsibilities

Design and implement end-to-end integrations of Grove with open-source AI frameworks, including Dynamo, llm-d, Ray, PyTorch, and related projects.
Build and maintain adapters, plugins, operators, and/or runtime components that enable Grove features across training and inference stacks.
Partner with framework owners to upstream changes, contribute patches, and ensure integrations remain maintainable.
Develop reference workflows, sample applications, and best-practice guides to accelerate adoption by users and partners.
Optimize performance, scalability, and reliability for distributed training and inference, including multi-node and multi-GPU environments.
Improve observability and operational readiness for Kubernetes-based deployments through metrics, logging, tracing, and debugging tools.
Participate in technical design reviews, define APIs and contracts, and ensure compatibility across framework and dependency versions.
Diagnose complex issues involving containers, networking, scheduling, CUDA/GPU utilization, and framework runtime behavior.