HPC SYSTEMS ENGINEER

University of Washington Medical Center•Seattle, WA

1d•$97,080 - $157,764•Hybrid

About The Position

UW Information Technology has an outstanding opportunity for an HPC Systems Engineer to join their team. Reporting to the Director, the HPC Systems Engineer is responsible for supporting the HPC efforts for the research computing group and related research computing technologies. This position also provides expertise and support to other endeavors when needed. This position requires a team-oriented professional, experienced in managing large IT systems for research using automated and software-defined approaches. This position regularly interfaces with other UWIT teams as well as research customers across campus. This position requires participation in a 24x7 on-call rotation. This is an essential position and is required to work remotely when the University suspends operations.

Requirements

Bachelors degree in computer science, information technology, scientific, engineering, or related field or experience
Minimum 4 years’ experience in Linux system administration experience or substantial experience working with Linux.
Basic knowledge of networking hardware, software, protocols, and concepts.
Demonstrated ability to work with minimal supervision, both independently and as part of a team.
Demonstrated excellent written/oral communication skills, technical documentation skills, user liaison skills, and personal interaction abilities.

Nice To Haves

Knowledge of Containerization platforms (e.g., Docker, Apptainer, LXC, Podman).
Web development skills such as Python and Django.
Security compliance experience such as NIST 800-171, CMMC L2, and FAR to protect data such as HIPAA, CUI, Export Controlled, etc.
Progressively responsible experience as an engineer, architect, or role with comparable technical responsibilities in a large Linux HPC environment.
Extensive experience with administration of Linux operating systems in a production environment, including experience with Red Hat Enterprise Linux or derivatives such as CentOS or Rocky Linux.
Familiarity with SLURM or other HPC scheduler (PBS Pro, PBS/Torque, SGE/UGE, LSF, etc).
Experience designing, configuring, and troubleshooting networks using both Ethernet and high-performance interconnects such as Infiniband (e.g. experience with VLAN, MLAG, and LACP configurations).
Proficiency in programming/scripting languages in the context of systems engineering or administration, preferably including Bash, Golang, or Python.
Experience in the configuration and use of mass deployment tools such as Warewulf, MAAS, Foreman, xCAT, Cobbler, or similar.
Proficiency with the use of Git for source control in collaboration with a team with multiple contributors.
Ability to administer and troubleshoot large high-performance parallel filesystems such as IBM Storage Scale (GPFS), Lustre, BeeGFS, Ceph.
Experience with the use of configuration management tools such as Ansible, SaltStack, or Puppet.
Experience with monitoring, analysis and visualization of metrics, telemetry, and logs (e.g. Grafana, Prometheus, splunc elasticsearch, rsyslog, etc).
Experience in a data center environment (e.g., racking equipment, running cables, labeling, asset tracking).
Experience with AI training and inference.
Experience with timeseries, relational, and non-relational database management or database query construction (e.g. influx, promql, redis, mongodb, postgres, mysql, nosql, mariadb, etc).
Scientific background, research experience, and/or experience in a University setting.

Responsibilities

Design, develop, adapt, integrate, manage, optimize, or deploy monitoring for new or existing systems, software, or automation to continually improve the performance, reliability, recoverability, manageability, and usability of research cyberinfrastructure.
Perform system management functions such as account administration, service administration, software installation and configuration, firmware updates, software updates, backing up and restoring data, improving system and service monitoring, or planning and execution of hardware replacements.
Monitor HPC and related systems health and performance, and troubleshoot and resolve issues that impact performance, health, or reliability.
Monitor support ticket and alerting queue; triage, respond to, and resolve tickets and on-call pages as appropriate.
Research and evaluate new and future technologies. Plan and architect new systems and the evolution and lifecycle of existing systems and clusters. Suggest or advise on new technologies to the Director of Research Computing Operations and other members of the Research Computing Operations team.
Effectively share experience and knowledge with other team members.
Other duties as assigned