HPC Engineer/Architect

Comspark group•New York, NY
•Hybrid

About The Position

The HPC Engineer/Architect will support day-to-day operations of large-scale parallel file systems, deploy and maintain Linux HPC infrastructure across multiple data centers, and assist HPC engineers and architects with operations and tickets. The role involves designing and overseeing Linux-based HPC clusters and storage, supporting research computing workflows, and maintaining the reliability and efficiency of the computational environment.

Requirements

  • 16 to 20 years of experience.
  • Experience working in a large-scale research-based HPC environment.
  • Proven experience working with distributed file storage solutions such as GPFS.
  • Experience deploying and troubleshooting Linux operating systems, including RHEL/CentOS.
  • Experience with scripting and automation, including Ansible, Python, and shell scripting.
  • Solid understanding of job schedulers, including LSF and Slurm.
  • Experience with GPU-based compute infrastructure, including CUDA.
  • Knowledge of Linux operating systems, parallel file systems (GPFS), job schedulers (LSF/Slurm), Ansible, Python, and shell scripting.

Responsibilities

  • Design, architect, and oversee implementation of Linux-based HPC clusters and storage.
  • Deploy physical hardware using HPC deployment tools and configuration and orchestration tools such as Ansible.
  • Tune, monitor, and troubleshoot GPFS parallel file systems.
  • Perform systems benchmarking and develop automated tests for the HPC environment.
  • Maintain and troubleshoot InfiniBand networks.
  • Automate and monitor the HPC user lifecycle process.
  • Install, configure, tune, and troubleshoot Slurm.
  • Plan, design, and implement the transition from the LSF scheduler to Slurm.
  • Manage the Slurm scheduler and translate research policies into scheduler configurations.
  • Consult with faculty and students to develop research pipelines for the HPC cluster.
  • Develop and maintain the user lifecycle software suite in Python and implement a CI/CD pipeline.
  • Test and automate upgrades of critical system applications using Ansible and shell scripts.
  • Support day-to-day operations of large-scale parallel file systems and Linux HPC infrastructure across multiple data centers.
  • Assist HPC engineers and architects with day-to-day operations and tickets.
  • Communicate effectively with clinicians, researchers, and other team members to develop technological solutions.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service