Senior HPC Engineer

Oak Ridge National Laboratory • Oak Ridge, TN

71d

About The Position

We are hiring a Senior Linux HPC Systems Engineer to design, operate and maintain clusters, servers, and workstations supporting services where science happens at ORNL! This position resides in the Emerging Technologies & Computing team in the Research Computing group in the Information Technology Services Directorate at Oak Ridge National Laboratory (ORNL). The Emerging Technology Computational Group facilitates ORNL goals through HPC systems engineering, integration, and support for the research community at ORNL. By providing design, deployment, optimization, monitoring, and tooling support across multiple clustered infrastructures, we facilitate Lab-wide R&D projects. Our HPC clusters range in scope from just a handful of nodes to over fifty-thousand cores. We partner with ORNL research organizations to enable research excellence and delivery. We work with other clustered computing and HPC groups to help research programs identify the best solutions for their needs. When we build our customer's environments, our team collaborates to design, implement, and maintain the systems from inception to retirement.

Requirements

• A BS degree in computer science, computer engineering, information technology, information systems, science, engineering, business, or a related discipline and a minimum of eight (8) to twelve (12) years of aligned professional experience is required for consideration.
• Masters and PhD degree holders in the same fields of study are also encouraged to apply.
• Five (5) or more years with managing UNIX/Linux Systems.
• Three (3) or years of proven experience with configuration management and automation tools such as Git, Jenkins, Ansible, or Puppet.
• Moderate proficiency in at least one scripting language such as Bash, Python, or others.
• Experience performing advanced troubleshooting and system administration with Linux Servers.
• Experience supporting large data systems.
• A strong desire to innovate and identify new technologies and opportunities and be able to communicate the potential benefits of those choices to others within the team and our research partners.
• A collaborative and upbeat approach to thrive on the opportunity to build trust and credibility and ultimately become a trusted advisor to our research teams.

Nice To Haves

• Active DOE Q, DoD Top Secret, or TS/SCI clearance is strongly preferred.
• Solid understanding of multiple operating systems and HPC cluster technologies.
• Experience with Rocky/CentOS/RHEL, Ubuntu, VMware.
• Understanding of HPC job schedulers (SLURM) and user support workflows.
• Experience with container technologies in HPC environments.
• Experience with multiple system deployment mechanisms (Warewulf, PXEboot, Cobbler, Bright).
• Experience with GPU clusters (NVIDIA, AMD) for AI/ML and scientific workloads.
• Deep expertise with high-performance parallel file systems (Lustre, GPFS/Spectrum Scale, BeeGFS, WEKA).
• Knowledge of storage networking (Infiniband, NVMe-oF, SAN/NAS architectures).
• Familiarity with RAID, ZFS, and object storage technologies.
• Strong background in performance monitoring, benchmarking, and I/O optimization.
• Experience with monitoring systems such as Grafana, CheckMK, Nagios, Zabbix, Ganglia.
• Previous experience working in a government, scientific, or other highly technical environment.
• Strong documentation skills and ability to prepare web-based documentation.

Responsibilities

• Advocate and promote HPC and clustered computing services to researchers who process large data sets and/or develop code as a part of their project.
• Ensure the availability, performance, scalability, and security of production systems.
• Leverage automation and monitoring solutions that minimize our day-to-day maintenance and scout opportunities to optimize system management practices or system performance.
• Provide strategic leadership in the design, deployment, and long-term evolution of HPC and clustered computing services, ensuring they align with ORNL's mission and scientific priorities.
• Set technical direction for HPC infrastructure initiatives, driving innovation in compute, storage, networking, and system management practices to support next-generation scientific workloads.
• Oversee the availability, performance, scalability, and security of mission-critical HPC production systems, balancing operational stability with research agility.
• Lead cross-functional collaborations with program POCs, research teams, and other HPC groups to architect solutions, optimize performance, and accelerate scientific discovery.
• Champion automation and advanced monitoring frameworks that reduce operational overhead, improve reliability, and provide predictive insights into system performance.
• Mentor and guide junior engineers and administrators, fostering professional growth and ensuring knowledge transfer within the HPC operations team.
• Represent ORNL in external collaborations, engaging with vendors, DOE, and peer institutions to influence technology roadmaps and adopt emerging HPC innovations.
• Develop and enforce best practices and standards for system deployment, monitoring, security, and lifecycle management across multiple HPC environments.
• Contribute to long-term planning, including capacity forecasting, technology evaluation, and adoption of advanced computing paradigms.
• Deliver ORNL's mission by aligning behaviors, priorities, and interactions with our core values of Impact, Integrity, Teamwork, Safety, and Service. Promote equal opportunity by fostering a respectful workplace - in how we treat one another, work together, and measure success.

Benefits

• Prescription Drug Plan
• Dental Plan
• Vision Plan
• 401(k) Retirement Plan
• Contributory Pension Plan
• Life Insurance
• Disability Benefits
• Generous Vacation and Holidays
• Parental Leave
• Legal Insurance with Identity Theft Protection
• Employee Assistance Plan
• Flexible Spending Accounts
• Health Savings Accounts
• Wellness Programs
• Educational Assistance
• Relocation Assistance
• Employee Discounts

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Industry: Professional, Scientific, and Technical Services
• Education Level: Bachelor's degree
• Number of Employees: 5,001-10,000 employees