Data Engineer Senior

ASM Research • Fairfax, VA

11d

About The Position

The Data Engineer Senior designs, builds, and maintains scalable data pipelines and core data infrastructure to ingest, process, store, and transform structured and unstructured data in a highly regulated client environment. This role develops and operates automated, end‑to‑end pipelines, optimizes data models and schemas, and ensures the reliability, scalability, and security of the enterprise data platform. The engineer also implements robust data quality, validation, and lineage mechanisms and collaborates with cross‑functional stakeholders to align data architecture with business and mission needs while supporting continuous improvement of data engineering practices.

Requirements

• Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent relevant experience in data engineering or software engineering.
• Typically 5–8 years of experience in data engineering or closely related roles, including hands‑on work with distributed data processing, data modeling, and ETL/ELT implementation.
• Demonstrated experience with modern data engineering frameworks and tools such as Apache Spark, Kafka or similar streaming platforms, and workflow orchestrators like Apache Airflow.
• Strong proficiency with relational and non‑relational data stores (e.g., PostgreSQL, cloud data warehouse platforms, object storage) and with designing schemas for analytics and integration use cases.
• Experience implementing data quality, validation, and lineage capabilities in production environments, including monitoring and alerting for data pipeline health.
• Ability to obtain and maintain a SECRET‑level background investigation, with U.S. citizenship required to support federal client environments.
• Solid understanding of data security principles, including encryption, access control, and secure handling of sensitive information in compliance‑driven environments.

Nice To Haves

• Experience designing and operating data platforms on major cloud providers such as AWS, Azure, or Google Cloud, including use of native data and analytics services.
• Professional certification in cloud data engineering or big data technologies (e.g., AWS Data Analytics, Google Professional Data Engineer, Azure Data Engineer Associate).
• Prior experience supporting mission‑critical or highly regulated government systems where data governance, auditability, and compliance requirements are central.
• Experience contributing to the evolution of enterprise data architectures, standards, and best practices, including documentation of data pipelines and participation in data governance forums.

Responsibilities

• Design and implement distributed data pipelines using frameworks such as Apache Spark, Kafka, or Flink to support high‑volume, near‑real‑time and batch data processing across the enterprise.
• Develop and optimize data models, schemas, and storage patterns across relational and non‑relational platforms (e.g., PostgreSQL, Snowflake, S3, data lakes) to enable analytics, reporting, and downstream system integration.
• Implement and operate ETL/ELT workflows using orchestration tools such as Apache Airflow or cloud‑native services, ensuring end‑to‑end automation, observability, and recoverability.
• Establish and maintain data quality validation rules, monitoring, and anomaly detection processes to protect data integrity across production pipelines.
• Build and maintain data lineage, cataloging, and metadata management solutions to support governance, auditability, and regulatory compliance in a highly regulated environment.
• Apply security best practices, including encryption, role‑based access controls, and compliance alignment, when handling sensitive and mission‑critical data.
• Troubleshoot and resolve complex pipeline failures, performance bottlenecks, and data inconsistencies, driving root‑cause analysis and long‑term remediation.
• Collaborate with data scientists, analysts, and application teams to translate analytical and integration requirements into scalable data engineering solutions and reusable data assets.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior