Senior Site Reliability Engineer - Storage

NVIDIA•Us, CA

15d•Hybrid

About The Position

NVIDIA is leading the way in groundbreaking developments in Artificial Intelligence, High-Performance Computing, and Visualization. The GPU, our invention, serves as the visual cortex of modern computers and is at the heart of our products and services. Our work opens up new universes to explore, enables unique creativity and discovery, and powers what were once science fiction inventions, from artificial intelligence to autonomous cars. NVIDIA is looking for phenomenal people like you to help us accelerate the next wave of artificial intelligence. Join our team at NVIDIA as a Senior Site reliability engineer focused on HPC storage and play a crucial role in designing, implementing, and optimizing on-prem High-Performance Computing (HPC) storage solutions while harnessing the power of cloud computing. You will be responsible for crafting and deploying distributed storage solutions, build automation tools, and ensuring the efficient operations of our growing IT ecosystem. You will collaborate closely with engineering teams to align infrastructure with their evolving needs, document best practices, and contribute to the success of ground breaking projects.

Requirements

BS in Computer Science (or equivalent experience) with 8+ years of relevant experience, MS with 5+ years of experience or Ph.D. with 3 years of experience
Deep experience with storage protocols such as nfs, NVMe/TCP, S3 and Lustre (LNet)
Experience with containerization technologies like Kubernetes and their integration with storage solutions
Proficiency in one or more programming languages (Python, GO) is a must.
Experience working with monitoring and configuration management tools such as Chef, Ansible, Puppet, Saltstack, etc
Background with cloud infrastructure - AWS, Azure or Google Cloud.
Experience with multiple monitoring stacks such as Prometheus+Grafana, Elasticsearch+Kibana.
Excellent communication and collaboration skills.

Nice To Haves

Knowledge of HPC and AI solution technologies from CPU’s and GPU’s to high speed interconnects and supporting software
Experience with RDMA (InfiniBand or RoCE) fabrics
Background with HPC cluster management tools such as Slurm, PBS, LSF, etc.
Passionate and experienced in AI methodologies.

Responsibilities

Design, implement an on-prem HPC infrastructure supplemented with cloud computing to support the growing IT needs of NVIDIA.
Design and implement advanced storage solutions, such as high-performance NFS, S3-compatible object storage, and distributed storage systems
Develop tooling to automate deployment and management of large-scale infrastructure environments, to automate operational monitoring and alerting, and to enable self-service consumption of resources.
Document the general procedures and practices, perform technology evaluations, related to distributed file systems.
Collaborate across teams to better understand developers' workflows and gather their infrastructure requirements.
Influence and guide methodologies for building, testing, and deploying applications to ensure optimal performance and resource utilization.