Big Data Architect

GIGATEC Engineering•Annapolis Junction, MD

11h

About The Position

In this key Software Engineer role, you'll be responsible for creating and maintaining data workflows and automation pipelines using Apache Airflow, keeping critical processes flowing as smoothly as a well-timed morning coffee. This role focuses on building reliable, scalable, and observable workflow orchestration solutions that support data engineering, analytics, and operational use cases, because no one wants a pipeline that is more decaf than dependable. You'll get to collaborate with data engineers, platform teams, and stakeholders to ensure workflows are efficient, secure, and production ready, delivering results with the consistency of a favorite espresso order.

Requirements

Experience using the Linux CLI and Linux tools
Experience developing Bash scripts to automate manual processes
Recent software development experience using Python and Java
Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows
Experience using Distributed Big Data processing engines including Apache Spark
Experience with containerization technologies such as Docker, containerd, and Podman
Experience with Git Source Control System
Familiar with AWS Cloud Services and Infrastructure
Experience using the Atlassian Tool Suite (JIRA, Confluence)

Nice To Haves

Fluent in sarcasm, kindness, and the occasional office banter

Responsibilities

Creating and maintaining data workflows and automation pipelines using Apache Airflow
Building reliable, scalable, and observable workflow orchestration solutions
Collaborating with data engineers, platform teams, and stakeholders to ensure workflows are efficient, secure, and production ready