Software Engineer 3 - Linux/Bash/Python/Apache Airflow/SQL/Jupyter Notebook/NumPy/JSON

Captivation Software • Annapolis Junction, MD

4d • $130,000 - $270,000

About The Position

Captivation Software is seeking a senior-level software engineer to support its mission-focused engineering work. The role focuses on dataflow design, data transport mechanisms, and Apache Spark-based distributed processing. The engineer will design, implement, and optimize data ingress and egress pathways to enable efficient, scalable, and reliable processing of analytics workloads. This position is open to direct hires only; candidates from third-party staffing or recruiting firms will not be considered.

Requirements

• Must currently hold a Top Secret/SCI U.S. Government security clearance with a favorable Polygraph.
• Must be a U.S. citizen.
• One of the following education and experience combinations: a master's degree in Computer Science or a related discipline from an accredited college or university and five years of SWE experience on programs and contracts of similar scope, type, and complexity; a bachelor's degree in Computer Science or a related discipline from an accredited college or university and seven years of such SWE experience; or nine years of such SWE experience.
• Experience using the Linux CLI and Linux tools.
• Experience developing Bash scripts to automate manual processes.
• Recent software development experience using Python and Java.
• Experience with Apache Airflow, including DAG design, scheduling, operators, and sensors.
• Experience with distributed big-data processing engines, including Apache Spark.
• Familiarity with SQL technologies such as MySQL, MariaDB, and PostgreSQL for querying, joining, and aggregating large datasets.
• Experience using Jupyter Notebook.
• Experience with data wrangling and preprocessing using tools such as pandas and NumPy.
• Experience working with structured, semi-structured, and unstructured data such as Parquet, JSON, CSV, and XML.
• Familiarity with data quality concepts, data validation, and anomaly detection.
• Experience with Git Source Control System.

Nice To Haves

• Familiarity with HPC job scheduling tools, including Slurm.
• Experience with the Atlassian Tool Suite, including Jira and Confluence.

Responsibilities

• Design, implement, and optimize data ingress and egress pathways.
• Support efficient, scalable, and reliable processing of analytics workloads.
• Use Apache Airflow to orchestrate, schedule, and monitor complex workflows.
• Develop Bash scripts to automate manual processes.
• Work with distributed big-data processing engines, including Apache Spark.
• Query, join, and aggregate large datasets using SQL technologies.
• Wrangle and preprocess data, and work with structured, semi-structured, and unstructured data.
• Apply data validation and anomaly-detection concepts.

Benefits

• Up to 20% 401(k) contribution, with no matching required and vesting from day one.
• $3,600 HSA contribution.
• Six weeks of paid time off.
• Company-paid employee medical, dental, and vision insurance.
• Company-paid life insurance, short-term and long-term disability insurance, and AD&D coverage.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior