Scientific Developer for Model Optimization

Axiom Consultants•College Park, MD

About The Position

Axiom is seeking a Scientific Developer to optimize the UFS Weather Model, including its newly integrated MPAS atmospheric dynamic core and coupled components. The role focuses on identifying and resolving performance bottlenecks; profiling large parallel applications across CPU and GPU performance, memory, cache, and disk I/O; and developing configurable, reproducible profiling workflows. The Scientific Developer will help integrate optimized, GPU- and accelerator-enabled components into the UFS build system, including configuring compilers, packages, and profiling or instrumentation. The role also involves identifying appropriate computing hardware, porting and testing accelerated code, ensuring portability and reproducibility within acceptable tolerances, and investigating bit-for-bit reproducibility for CPU and GPU optimizations. The developer will contribute technical and strategic documentation, collaborate with teams supporting models and forecast workflows, and provide technical guidance and mentoring. The role also includes exploring and validating the use of generative AI and large language models to improve code quality and development efficiency.

Requirements

Bachelor's or master's degree in meteorology, atmospheric science, physical sciences, mathematics, computer science, software engineering, scientific computing, or a closely related field, with demonstrated numerical modeling or closely related scientific computing experience.
Demonstrated experience in high-performance computing, software optimization, and building, running, debugging, and profiling large-scale parallel applications.
Proficiency in C, C++, and/or Fortran, as well as MPI.
Familiarity with GPU or heterogeneous programming and/or portability frameworks such as CUDA, OpenACC, OpenMP target offloading, Kokkos, RAJA, PyTorch, or JAX.
Familiarity with profiling concepts and/or tools such as gprof, TAU, or Scalasca.
Experience with Linux/Unix shell scripting and automation.
Demonstrated proficiency with Git or similar tools for code collaboration and management.
Demonstrated ability to apply software engineering concepts and best practices.

Nice To Haves

Demonstrated experience with build systems such as CMake, Meson, autotools, or make; strong proficiency in configurable scripting and automation is acceptable instead.
Ph.D. in a quantitative field with research computing experience.
Direct experience with the Unified Forecast System (UFS), MPAS, or other atmospheric modeling systems; familiarity with NWP forecast workflows is beneficial.
Experience with workflow tools such as ecFlow, Rocoto, or Cylc and package management tools such as Spack or pip in HPC environments.
Familiarity with leveraging generative AI or large language model tools to optimize and assist software development workflows.
Familiarity with continuous integration workflows and frameworks.

Responsibilities

Identify and resolve performance bottlenecks in the UFS Weather Model, including the integrated MPAS atmospheric dynamic core and coupled components.
Identify model performance improvements through code optimization, build configuration, and heterogeneous and portable computing, particularly with CPUs and GPUs.
Implement runtime profiling for large parallel applications, including CPU and GPU performance, memory, cache, and disk I/O; develop configurable, reproducible profiling workflows.
Enable integration of optimized, GPU- or accelerator-enabled, and profiling-enabled components into the UFS build system, including configuration of compilers, packages, and profiling or instrumentation.
Identify computing hardware resources and port and test accelerated code on appropriate systems, ensuring portability and reproducibility within acceptable tolerance.
Investigate bit-for-bit reproducibility for CPU and GPU optimizations and propose and implement appropriate tests.
Contribute to strategic, technical, and other documentation about profiling, instrumentation, optimization, and GPU or portable programming.
Collaborate with other teams to support models, forecast workflows, and other functions by delivering optimized code and profiling or instrumentation frameworks.
Provide technical guidance and mentoring to colleagues on profiling, instrumentation, optimization, and GPU or portable code development.
Explore and validate the use of generative AI and large language models to improve code quality and development efficiency.