HPC Systems Engineer

University Corporation for Atmospheric Research • Boulder, CO

13h • $68,040 - $85,050

About The Position

As part of NCAR’s High-Performance Computing Systems Group (HSG), the HPC Systems Engineer performs systems engineering and technical support for CISL high-performance computational and storage systems, archival systems, high-performance networks, and data transfer services. The multi-vendor environment includes specialized hardware and requires coordination with other groups and stakeholders. Production systems are located at the NCAR Wyoming Supercomputing Center (NWSC) in Cheyenne, Wyoming. The engineer must work at NWSC during some system installations, upgrades, and troubleshooting periods and may travel there occasionally from the Boulder area.

The role includes installing, integrating, maintaining, administering, troubleshooting, and operating hardware and software, including petascale computational systems, scheduling environments, high-speed networks, and storage systems. It uses system management, deployment, monitoring, resource-control, data-gathering, data-transfer, data-management, archiving, cloud, and network services, as well as performance-tuning and benchmarking tools. The engineer routinely monitors system behavior, identifies abnormal conditions, takes remedial action, and may diagnose or repair failed hardware. The role provides advanced on-call support for system problems and contributes to reporting, meetings, and representation of the group.

The position requires a periodic 24x7 on-call rotation with other staff. The stated hiring range is $68,040–$85,050; final salary depends on education, experience, and relevant skills. The position is regular, and visa sponsorship and relocation assistance are not available.

Requirements

• Bachelor’s degree in a computer-related or engineering field, or an equivalent combination of education and relevant experience; an Associate’s degree plus four years of work experience is given as an example.
• Skill with multiple operating systems, including Linux, and with scripting and programming languages.
• Experience with Linux, particularly networking, storage, performance, and installation and management in a Linux-based supercomputer environment.
• Experience with Linux scripting languages and at least one higher-level programming language.
• Experience with a job scheduling system such as PBS or SLURM.
• Experience with clustered file systems such as Spectrum Scale or Lustre and clustered system management tools and techniques.
• Experience with at least one type of HPC storage, such as block, object, cloud, or tape.
• Experience with version control and configuration management systems.
• Experience with disaster recovery of single-image systems and clusters.
• Ability to serve in a periodic 24x7 on-call rotation with other staff.
• Ability to travel occasionally to the NCAR Wyoming Supercomputer Center, approximately 90 miles north of Boulder.

Nice To Haves

• Skill or familiarity with HPC and HPS software analysis, design, and development.
• Skill or familiarity with HPC and HPS networking, high-performance interconnects, and security.
• Skill or familiarity with installation and integration of HPC and/or HPS hardware and software.
• Skill or familiarity with developing technical specifications for RFPs, RFIs, and RFQs.
• Skill or familiarity with upgrading the hardware and software stack on HPC and HPS systems.

Responsibilities

• Perform systems engineering and technical support for high-performance computational, storage, archival, and network systems and data transfer services.
• Install, integrate, maintain, administer, troubleshoot, and operate hardware and software systems, including petascale computing systems, scheduling environments, high-speed networks, and storage systems.
• Use system management, deployment, monitoring, resource-control, statistical data-gathering, data-transfer, data-management, archiving, cloud, and network services and tools for performance tuning and benchmarking.
• Routinely operate and monitor systems and related hardware to ensure proper and efficient operation.
• Alert group staff, vendor representatives, and/or CISL Operations staff to abnormal conditions and take remedial action as necessary.
• Diagnose and potentially repair failed hardware components.
• Provide advanced 7x24 on-call troubleshooting and problem resolution for users, CISL sections, and vendor-employed engineers and analysts; consult other staff as appropriate.
• Provide regular reports on HPC Systems Group activities to management and potentially contribute to the NCAR Annual Scientific Report and development plans.
• Attend group, divisional, and lab meetings and represent the HPC Systems Group at organizational meetings, user groups, conferences, and planning meetings as needed.
• Work at the NWSC during periods of system installation, upgrade, or troubleshooting and travel there occasionally.

Benefits

• Health insurance, including medical, dental, vision, and short- and long-term disability insurance.
• Retirement plan; UCAR contributes 10% of base salary.
• Leave plans, including a minimum of 20 days of annual PTO and 10 paid holidays.
• 12 weeks of paid parental leave.
• FAMLI Leave.
• Tuition assistance program.
• Employee assistance programs.
• Public Service Loan Forgiveness (PSLF)-eligible employer.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level