HPC Infrastructure Engineer

Oak Ridge National Laboratory•Oak Ridge, TN

23h

About The Position

The High-Performance Computing Systems Section within the National Center for Computational Sciences (NCCS) is seeking an HPC Infrastructure Engineer to join the HPC Infrastructure group. NCCS provides advanced computational and data science infrastructure and technical and scientific expertise for research across a broad range of scientific domains. It hosts the Oak Ridge Leadership Computing Facility, a Department of Energy National User Facility that operates Frontier, the nation’s first exascale supercomputer. The engineer will deploy, configure, administer, monitor, and troubleshoot HPC-scale Linux services, primarily on Red Hat and Rocky Linux. The role also designs and manages infrastructure using infrastructure-as-code and configuration-management tools, deploys resources in the NCCS virtualization environment, and identifies and leads automation efforts. The engineer will work with the NCCS Kubernetes Platform team to deploy and manage services, develop standardized workflows and CI/CD pipelines, and maintain internal tools written in languages such as Python and Bash. The position leads small infrastructure projects, mentors and trains junior staff, and proposes and implements improvements to infrastructure systems and team processes. The role requires the ability to obtain and maintain a Department of Energy clearance and is designated for Workplace Substance Abuse Program testing, including a pre-placement drug test and ongoing random testing.

Requirements

Bachelor’s degree in computer science or a closely related field and at least 5 years of experience in Linux systems and Kubernetes platform administration; or a master’s degree and at least 4 years of such experience. An equivalent combination of education and experience will be considered.
Ability to obtain and maintain a Department of Energy clearance.
Ability to pass a pre-placement drug test and participate in an ongoing random drug-testing program, as this is a WSAP-designated position.

Nice To Haves

Excellent interpersonal and communication skills and the ability to work as part of a team.
Strong experience designing, building, and maintaining Kubernetes platform tools.
Strong working knowledge of Linux system fundamentals and common network protocols.
Programming and scripting skills in common languages such as Python and Bash.
Understanding of versioning and code-review tools such as GitHub and GitLab.
Experience implementing and supporting highly available systems and services.
Experience with configuration-management tools such as Puppet or Ansible.
Experience deploying and maintaining virtual environments using VMware.
Experience deploying, maintaining, and troubleshooting infrastructure services such as OpenLDAP, DNS, and DHCP.
Ability to plan, prioritize, and complete assigned projects with minimal supervision.

Responsibilities

Deploy, configure, and manage HPC-scale services in a Linux environment, primarily Red Hat and Rocky Linux.
Perform regular patches, updates, and backups.
Monitor systems using tools such as Nagios and Grafana.
Respond to and assist with troubleshooting issues.
Design, deploy, and manage the infrastructure platform using infrastructure-as-code tools such as Terraform, Ansible, and Puppet.
Deploy and manage resources in the NCCS virtualization environment.
Identify potential automation targets and lead efforts to automate processes.
Define automation and configuration-management policies and procedures for the team and organization.
Work with the NCCS Kubernetes Platform team to deploy and manage services for the HPC Infrastructure group.
Design standardized and automated workflow patterns and build and maintain CI/CD pipelines.
Develop, maintain, and review high-quality code for internal tools using languages such as Python and Bash.
Lead small infrastructure projects through their lifecycle.
Mentor and train junior staff, create training documentation, hold knowledge-sharing sessions, and foster skill growth across the team.
Propose and implement improvements to existing infrastructure systems, as well as new systems, processes, and procedures.