Senior Parallel File System Software Developer

LLNL•Livermore, CA

1d•$210,630 - $267,060•Hybrid

About The Position

We have an opening for a Senior Parallel File System Software Developer to provide technical leadership in researching, architecting, developing, maintaining, and integrating software and hardware solutions that underpin scalable storage services within the Livermore Computing high-performance computing center. In this role, you will apply subject-matter expertise and independent technical judgment to advance and sustain production parallel file systems: including resolution of highly complex, system-wide software defects and architectural issues in partnership with system administration staff and external collaborators. You’ll independently develop and lead the technical evaluation, design, integration, and sustainment of software fixes, upstream patches, and locally developed enhancements to address challenging issues on large-scale Linux systems. Additionally, this position will establish technical direction for innovative software capabilities in Lustre and ZFS file systems. This position is in the Livermore Computing Division within the Computing Principal Associate Directorate. This position offers hybrid option. You will have the flexibility to remotely work from home with regular visits to LLNL.

Requirements

Ability to maintain a U.S. DOE Q-level security clearance which requires U.S. citizenship.
Bachelor’s degree in computer science or related field or the equivalent combination of education and related experience.
Significant and demonstrated subject-matter expertise in parallel file-system internals, including substantial experience with the Lustre codebase.
Significant experience in a production high-performance computing environment where unplanned downtime has significant operational consequences and end-user impact.
Advanced proficiency developing systems software in C in a team environment with additional proficiency in C++, Rust, or Python.
Highly advanced knowledge and technical experience with Linux operating systems and Linux kernel interfaces, including configuration, networking, and system security.
Advanced problem-solving and debugging skills to diagnose multi-component problems in highly parallel, multi-threaded systems and identify the root cause of complex storage system issues — includes advanced proficiency with Linux debugging and inspection tools (examples: strace, perf, gdb, the /proc file system, and/or eBPF).
Expert verbal, written, interpersonal, and technical-influence skills necessary to advise management, collaborate across organizations, communicate with external technical communities, and present and explain complex technical information under limited direction.
Ability to set priorities, independently resolve complex problems, and apply new technologies to broadly defined tasks and projects in a fast-paced environment.
Demonstrated ability to lead technical projects or major workstreams, establish technical direction, and influence decisions across multidisciplinary teams.
Availability to work off-hours to resolve production problems, on an as-needed basis.

Nice To Haves

Master’s degree in computer science or related field or the equivalent combination of education and related experience.
Familiarity with open-source storage community contributions — cherry-picking patches, umbrella organizations (such as OpenSFS), upstreaming contributions (such as to the Lustre mainline), etc.
Demonstrated record of technical contributions to open-source storage projects, technical publications, conference presentations, or leadership in relevant technical communities.
Experience with integration of traditional storage systems and cloud-first technologies such as S3 data transfer protocol, object storage, OIDC/OAuth, and/or metadata extraction and cataloging systems.
Experience defining storage architecture or technical roadmaps for large-scale HPC, scientific computing, or similarly mission-critical environments.

Responsibilities

Provide software engineering technical leadership for the evolution, reliability, and performance of production Lustre file systems running at petabyte and billion-object scale.
Serve as a technical subject matter expert and provide technical leadership for highly complex storage-system software development initiatives for new features, performance improvements, and maintenance of Lustre codebases along with related projects (ZFS, Lustre monitoring tools, storage quota systems, etc.).
Review both upstream and local code changes, set technical criteria for adoption, and lead integration of upstream and locally developed changes into production Lustre deployments.
Lead cross-functional efforts to identify, integrate, and implement innovative storage system solutions, including evaluating technical tradeoffs, risks, lifecycle impacts, and alternative architectures.
Lead diagnosis and resolution of critical system-wide performance, scalability, and reliability issues in production storage systems, where decisions may affect availability of major Laboratory computing resources and mission workloads.
Establish and influence technical roadmaps, architecture, and engineering priorities for parallel file-system software in alignment with long-range Livermore Computing and Laboratory mission needs.
Provide strategic technical guidance to project stakeholders, management, and partner organizations, including recommendations that influence operational, technology-investment, and service-delivery decisions.
Mentor and develop technical staff across the organization and share expertise broadly with the next generation of storage professionals.
Perform other duties as assigned.