HPC Linux System Administrator
About The Position
MRI Technologies is hiring an HPC Linux System Administrator to support NASA under the JETS II contract. The role supports the Flight Sciences Laboratory (FSL) at NASA's Johnson Space Center, whose high-performance computing environment serves NASA human spaceflight programs and more than 1,000 engineers and scientists. The cluster comprises more than 700 machines, 26,000 cores, and over 10 petabytes of storage. This is a hands-on, on-premises HPC administration role focused on operating and improving the cluster. The administrator will manage the job scheduler and parallel filesystem, help users run software environments using HPC containers, and build CI/CD workflows that run on cluster run nodes, including Jacamar. The position is based onsite at NASA Johnson Space Center in Houston, Texas. Fully remote work may be considered for candidates with strong, demonstrated HPC administration experience.
Requirements
- Production experience administering an HPC job scheduler such as Slurm, PBS/Torque, or LSF; experience limited to submitting jobs is not sufficient.
- Production experience administering a high-speed parallel filesystem such as Lustre or GPFS/Spectrum Scale; general NAS, SAN, or NFS storage administration does not substitute.
- At least 5 years of Linux system administration in on-premises, bare-metal, or cluster/research-computing environments, including at least 2 years of HPC administration.
- A bachelor's degree or equivalent certification in a related field, typically with a minimum of 5 years of experience.
- Experience using containers such as Apptainer/Singularity or Podman in an HPC setting to package and run user environments on a shared cluster.
- Experience building and supporting CI/CD workflows.
- Experience with system configuration management.
- Experience with system monitoring and alerting.
- Strong problem-solving, planning, and communication skills.
- Ability to work collaboratively as part of a team.
- Must provide proof of U.S. Citizenship or U.S. Permanent Residency and complete a U.S. government background investigation.
Nice To Haves
- Experience with Red Hat-based Linux distributions such as RHEL, Rocky, or Alma.
- Familiarity with InfiniBand high-speed networking.
- Experience with cluster provisioning tools such as xCAT or Warewulf.
- Experience with Ansible and/or Foreman for configuration management.
- Experience with log consolidation and monitoring tools, and with Git/GitLab, including CI/CD pipelines.
- Familiarity with Jacamar or comparable tooling for running CI/CD jobs against HPC clusters and run nodes.
- Experience applying AI in system administration, such as integrating AI tooling into operational workflows, automation, or user-facing HPC jobs.
- MPI workflow and administration experience, such as Open MPI, MPICH, or Intel MPI.
- Package management and environment modules experience, such as Lmod/Environment Modules, SPACK, or EasyBuild.
- Knowledge of NASA security mechanisms, including security plans, POAMs, ATOs, and Risk Assessments.
Responsibilities
- Administer and improve the HPC cluster supporting NASA's Flight Sciences Laboratory.
- Administer the HPC job scheduler, including partition and QOS configuration, accounting, node states, and troubleshooting failed or stuck jobs.
- Administer a high-speed parallel filesystem such as Lustre or GPFS/Spectrum Scale.
- Help users package and run software environments, including older operating system or package versions, on the shared cluster using HPC containers.
- Build and support CI/CD workflows, including workflows that run on HPC clusters and run nodes, such as those using Jacamar.
