Data Engineer

Defense Logistics Agency•Battle Creek, MI

1d•Hybrid

About The Position

This position is for a Data Engineer. The role involves architecting, developing, testing, and maintaining scalable data pipelines to ingest structured and unstructured data from various sources using complex ETL/ELT processes. The engineer will develop resilient batch and real-time data solutions using Python, Scala, or Java, and frameworks like Apache Spark and Airflow. Responsibilities include implementing CI/CD practices, troubleshooting pipeline executions, and optimizing performance, integrity, and cost-effectiveness for DLA systems. The role also entails leading the design and administration of data warehouse and data lake structures, administering and optimizing cloud data warehouses (Snowflake, Redshift, or BigQuery), enforcing data governance and quality, and serving as a principal technical advisor for the AI and Analytics CoE. Additionally, the position involves translating data requirements, providing guidance on data architecture best practices, mentoring junior staff, establishing coding standards, communicating complex technical concepts to non-technical stakeholders, and researching/prototyping emerging data technologies. The engineer will also lead technical planning and migration strategies to modernize legacy systems and transition data workloads to cloud-native platforms.

Requirements

Must be a U.S. citizen
One year of specialized experience that equipped you with the particular competencies to successfully perform the duties of the position and is directly in or related to this position.
To qualify at the GS-14 level, applicants must possess one year of specialized experience equivalent to the GS-13 level or equivalent under other pay systems in the Federal service, military, or private sector.
Applicants must meet eligibility requirements including time-in-grade (General Schedule (GS) positions only), time-after-competitive appointment, minimum qualifications, and any other regulatory requirements by the cut-off/closing date of the announcement.
Creditable specialized experience includes: mastery in designing and maintaining enterprise data pipelines optimizing cloud data warehouses providing technical architecture guidance.
Selectees are required to have a REAL ID or other acceptable identification documents to access certain federal facilities.

Responsibilities

Architects, develops, tests, and maintains scalable data pipelines to ingest structured and unstructured data from disparate DLA, DoD, and commercial sources using complex ETL/ELT processes.
Develops resilient batch and real-time data solutions using Python, Scala, or Java, and frameworks like Apache Spark and Airflow for strategic reporting and operational AI model serving.
Implements CI/CD practices for automated testing and deployment, troubleshooting pipeline executions to optimize performance, integrity, and cost-effectiveness for DLA systems.
Leads the design and administration of data warehouse and data lake structures, implementing dimensional and data vault models to support business intelligence and AI/ML use cases.
Administers and optimizes cloud data warehouses such as Snowflake, Redshift, or BigQuery, managing partitioning, indexing, and clustering strategies for high-performance analytics.
Enforces data governance, data quality, and enterprise data catalogs, collaborating with security teams to apply access controls, encryption, and data masking for CUI and Secret data.
Serves as principal technical advisor for the AI and Analytics CoE, translating data requirements from data scientists, ML engineers, and analysts into technical specifications for logistics systems.
Provides guidance on data architecture best practices, mentors junior technical staff, establishes coding standards, and conducts peer reviews for data engineering projects.
Communicates complex technical concepts regarding data architecture and pipeline design to non-technical stakeholders and senior leadership to guide strategic organizational decisions.
Researches, evaluates, and prototypes emerging data technologies and architectural patterns, conducting technical assessments and proofs-of-concept for cloud services and processing engines.
Leads technical planning and migration strategies to modernize legacy systems, transitioning data workloads to cloud-native platforms while maintaining expertise in industry trends.