Senior Data Pipeline Engineer
About The Position
We are seeking a results-driven ETL & Data Process Automation Engineer whose primary focus is transforming reporting workflows through cutting-edge process automation. In this role, you will design, engineer, and deploy highly scalable Extract, Transform, Load (ETL) data pipelines that handle both batch and incremental data ingestion with robust validation controls. Utilizing SQL and Python data frameworks (Pandas, PySpark, or Polars), you will preprocess, normalize, aggregate, and reshape high-volume datasets into clean, actionable structures. Furthermore, you will develop and maintain reusable Python automation packages for agency-wide use, driving seamless data processing and automated reporting across critical mission threads.
Requirements
- ACTIVE SECURITY CLEARANCE AT THE TS/SCI POLYGRAPH LEVEL IS REQUIRED
- Proven expertise designing and automating end-to-end reporting processes, replacing manual churn with programmatic pipelines.
- Deep experience building robust batch and incremental ETL/ELT pipelines with automated data validation, quality checks, and error handling.
- Mastery of SQL and Python for preprocessing, filtering, normalizing, aggregating, and reshaping complex, multi-source datasets.
- Hands-on proficiency with modern Python data libraries, specifically Pandas, PySpark, or Polars.
- Experience authoring, testing, and maintaining reusable Python packages and libraries for deployment and use by agency personnel.
- Bachelor’s Degree in Data Science, Computer Science, Computational Linguistics, Mathematics, or a related technical discipline, PLUS 10 + years of professional experience in data science, NLP, or software engineering OR Associate's Degree PLUS 12+ years of specialized technical experience in lieu of a degree.
Nice To Haves
- Hands-on experience constructing and automating interactive visual dashboards using Power BI, Tableau, or similar business intelligence tools.
- Familiarity operating within distributed big-data environments using Apache Spark or Hadoop ecosystems.
- Exposure to automated testing and deployment pipelines (GitLab CI, Jenkins) for python packages and data scripts.
Responsibilities
- Design, engineer, and deploy highly scalable Extract, Transform, Load (ETL) data pipelines that handle both batch and incremental data ingestion with robust validation controls.
- Utilize SQL and Python data frameworks (Pandas, PySpark, or Polars) to preprocess, normalize, aggregate, and reshape high-volume datasets into clean, actionable structures.
- Develop and maintain reusable Python automation packages for agency-wide use, driving seamless data processing and automated reporting across critical mission threads.
- Automate end-to-end reporting processes, replacing manual churn with programmatic pipelines.
- Build robust batch and incremental ETL/ELT pipelines with automated data validation, quality checks, and error handling.
- Preprocess, filter, normalize, aggregate, and reshape complex, multi-source datasets using SQL and Python.
- Author, test, and maintain reusable Python packages and libraries for deployment and use by agency personnel.
Benefits
- discretionary bonus compensation
- comprehensive benefits package
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
Associate degree
