Research Scientist, Distributed Workflows

Oak Ridge National Laboratory•Oak Ridge, TN

7d•Hybrid

About The Position

Oak Ridge National Laboratory (ORNL) is seeking a Research Scientist to advance distributed workflow orchestration and cross-facility integration in support of the Genesis Mission, the American Science Cloud, and the Oak Ridge Leadership Computing Facility (OLCF). Working within the Workflows and Ecosystem Services (WES) Group at the National Center for Computational Sciences (NCCS), you will design and deploy APIs, service meshes, and workflow orchestration systems that bridge leadership computing resources, instruments, and cloud platforms. You will collaborate with scientists, facility operators, and DOE laboratory partners to lower barriers to scalable, automated scientific computing. WES designs and implements workflow, data, and ecosystem technologies that enable reliable, automated execution of scientific applications across heterogeneous, geographically distributed computing environments, integrating compute, storage, networking, and visualization resources into cohesive, end-to-end systems. The WES Group sits within the Advanced Technologies section of the National Center for Computational Sciences (NCCS), a division of the Computing and Computational Sciences Directorate (CCSD) at ORNL. NCCS provides state-of-the-art computational and data science infrastructure for technical and scientific professionals and hosts the Oak Ridge Leadership Computing Facility (OLCF), a DOE Office of Science User Facility that is home to Frontier, the first supercomputer to break the exascale barrier. NCCS staff work at the forefront of leadership computing, enabling scientific discovery for researchers across government, academia, and industry. As a U.S. Department of Energy (DOE) Office of Science national laboratory, ORNL has an extraordinary 80-year history of solving the nation’s biggest problems. We have a dedicated and creative staff of over 7,000 people! Our vision for diversity, equity, inclusion, and accessibility (DEIA) is to cultivate an environment and practices that foster diversity in ideas and in the people across the organization, as well as to ensure ORNL is recognized as a workplace of choice. These elements are critical for enabling the execution of ORNL’s broader mission to accelerate scientific discoveries and their translation into energy, environment, and security solutions for the nation.

Requirements

Ph.D. in Computer Science, Computer Engineering, or a closely related field.
At least 2 years of experience designing or operating distributed systems, APIs, or workflow orchestration frameworks.
At least 1 year of programming experience in Python, with working familiarity in Bash and/or a compiled language (C/C++ or similar).
Experience with Linux-based HPC environments and familiarity with job scheduling systems (e.g., SLURM or PBS).
Strong verbal and written communication skills, with the ability to collaborate across technical and scientific teams.

Nice To Haves

Demonstrated experience publishing research in high-impact venues such as SC, IPDPS, HPDC, PEARC, or IEEE eScience.
Familiarity with RESTful API design, containerization (Docker/Singularity/Apptainer), and microservice architectures.
Experience working in or coordinating across DOE national laboratory environments, particularly in the context of integrated or cross-facility research infrastructure.
Background in scientific data management, provenance tracking, or metadata systems.
Demonstrated ability to lead technical projects, manage milestones, and coordinate multidisciplinary teams.
Familiarity with basic cyber-security principles (e.g., SSH key hygiene, least privilege, network segmentation) as applied to workflow and API design.

Responsibilities

Distributed Workflow Orchestration: Architect, implement, and operate cross-facility workflow orchestration systems that integrate job schedulers (e.g., SLURM), data movement services (e.g., Globus), and computing endpoints across multiple DOE facilities. Design execution patterns that support autonomous experimentation, multi-facility data pipelines, and event-driven automation.
API and Service Mesh Development: Design and implement RESTful APIs and service meshes that expose OLCF computing resources to diverse scientific workflows and external facilities. Define API specifications, manage versioning, and guide systems from prototype through production deployment.
Genesis Mission / American Science Cloud Integration: Lead ORNL’s technical engagement with the Genesis Mission, architecting cross-facility workflow and data infrastructure that connects leadership computing, experimental facilities, and cloud platforms in support of DOE’s vision for integrated science. Drive contributions to the American Science Cloud (AmSC) by designing and deploying interoperable APIs, data movement services, and orchestration layers that enable seamless scientific workflows across the DOE complex. Coordinate with partner laboratories and facilities (e.g., ANL, NERSC) to align interfaces and standards, and establish ORNL/OLCF as a foundational compute and data hub within the emerging national science infrastructure.
Agentic Workflow Design and Autonomous Control Systems: Design and implement agentic workflow architectures that enable autonomous, closed-loop scientific experimentation across HPC and instrument facilities. Develop LLM-integrated orchestration layers, tool-use pipelines, and event-driven control systems that allow AI agents to plan, execute, monitor, and adapt multi-step scientific workflows with minimal human intervention. Apply agentic patterns to use cases such as autonomous materials discovery, self-steering simulations, and adaptive experimental campaigns, in coordination with domain scientists and facility operators.
Scientific Domain Support: Collaborate with domain scientists in areas such as climate science, materials science, and autonomous experimentation to translate research objectives into scalable, automated workflow solutions. Provide documentation, training, and direct user support.
Scripting and Tooling: Develop command-line tools and automation in Python, Bash, and/or C/C++ to encapsulate workflow steps, manage configuration files (e.g., YAML/JSON), and implement robust logging, error handling, and checkpoint/retry strategies.
Operational Reliability and Optimization: Diagnose job failures, mitigate bottlenecks, and improve throughput, latency, and resource utilization. Use scheduler and Linux tools (e.g., sacct, squeue, coreutils, ssh, tmux, top, iostat) to monitor, analyze, and tune workflows.
Research and Publication: Conduct original research on workflow systems, orchestration, and data management. Publish findings in high-impact, peer-reviewed venues and present work at major conferences (e.g., SC, HPDC, PEARC, eScience).