Senior Data Engineer

Ad Hoc•Columbia, MD

13d•Remote

About The Position

Ad Hoc is a technology company that empowers organizations to deliver scalable, impactful digital services. Using modern, agile methods, our team creates products that meet people’s needs and transform their experience of government. We partner with agencies to build new capabilities, deliver products, establish data as a strategic asset for informed decision-making, modernize legacy systems, and build the digital service infrastructure necessary to scale their mission impact. The Federal Civilian business unit supports many customers spanning the federal, commercial, and nonprofit space. Our customers include NASA, the General Services Administration, Office of Personnel Management, the Library of Congress, Health & Human Services, and the FDIC.

Requirements

Bachelor's degree in Computer Science, Information Systems, Engineering, Mathematics, or a related technical discipline with 7+ years of professional experience. Relevant years of experience may be substituted for formal education.
7+ years of experience designing, building, and maintaining enterprise-scale data platforms and data pipelines.
Strong proficiency in SQL with experience developing, optimizing, and troubleshooting complex queries and large datasets.
Professional experience developing software and data solutions using Python, Java, Scala, or a comparable programming language.
Experience designing, developing, and maintaining ETL/ELT pipelines for batch and real-time data processing.
Experience working with cloud-based data platforms and services within AWS, Microsoft Azure, or Google Cloud Platform.
Experience with modern cloud data platforms such as Snowflake, Databricks, Amazon Redshift, Google BigQuery, or Azure Synapse Analytics.
Experience with relational database technologies such as PostgreSQL, SQL Server, Oracle, or MySQL, along with familiarity with NoSQL database solutions.
Experience with distributed data processing frameworks such as Apache Spark or equivalent big data technologies.
Strong understanding of data modeling, dimensional modeling, schema design, data warehousing, and database optimization.
Experience implementing data quality, validation, governance, metadata management, and data lineage best practices.
Experience working with structured, semi-structured, and unstructured data from multiple sources.
Experience with version control systems such as Git and CI/CD practices supporting data engineering workflows.
Understanding of data security, privacy, encryption, and access control principles.
Experience monitoring, troubleshooting, and optimizing production data systems for scalability, availability, and performance.
Experience working within Agile software development environments and collaborating across cross-functional engineering teams.
Strong analytical, troubleshooting, and problem-solving skills with the ability to make sound technical decisions.
Excellent written and verbal communication skills with the ability to explain technical concepts to both technical and non-technical audiences.
Demonstrated ability to mentor junior engineers through code reviews, technical guidance, and knowledge sharing.
Ability to obtain and maintain a U.S. Public Trust clearance.

Nice To Haves

Experience supporting U.S. Federal Government programs or other highly regulated environments.
Experience with orchestration platforms such as Apache Airflow, Prefect, Azure Data Factory, or AWS Step Functions.
Experience with streaming technologies such as Apache Kafka, Amazon Kinesis, or Azure Event Hubs.
Experience using dbt or other modern data transformation frameworks.
Experience with Infrastructure as Code tools such as Terraform or AWS CloudFormation.
Experience working with Docker and Kubernetes in cloud-native environments.
Experience building analytics and business intelligence solutions using Tableau, Power BI, Looker, or similar visualization tools.
Existing Public Trust or higher security clearance.
Experience serving as a technical lead or mentoring engineers on enterprise-scale data initiatives.

Responsibilities

Serve as an experienced individual contributor within a team, with the expectation that you will continue to develop your leadership, technical guidance, and mentoring skills.
With minimal oversight from leadership, design, build, and optimize scalable data solutions that support business and customer needs while ensuring projects meet scope, schedule, and delivery commitments.
Contribute to the long-term data strategy of the program, influence architectural decisions, and collaborate closely with software engineers, product managers, analysts, architects, and stakeholders to deliver reliable, secure, and scalable data platforms.
Serve as the primary data engineering lead for initiatives and utilize strong leadership and communication skills to drive improvements in data engineering processes, platform reliability, and engineering best practices.
Evaluate and recommend multiple technical approaches to solve complex data engineering and architecture challenges.
Design, develop, maintain, and optimize scalable data pipelines supporting both batch and real-time processing.
Design and implement robust ETL/ELT workflows that transform raw data into reliable, consumable datasets.
Build and maintain scalable data models, data warehouses, and cloud-native data architectures.
Develop solutions for structured, semi-structured, and unstructured data sources.
Generate data architecture recommendations and successfully implement approved solutions.
Ensure data quality, integrity, governance, lineage, security, and observability across data platforms.
Optimize database performance, query execution, storage strategies, and overall system scalability.
Diagnose and resolve production issues while implementing long-term improvements to increase system reliability and performance.
Collaborate with cross-functional teams to translate business requirements into scalable technical solutions.
Present technical designs, architecture diagrams, and implementation strategies to clients, stakeholders, partners, and engineering teams.
Champion data engineering best practices, coding standards, automation, and operational excellence.
Mentor junior engineers through technical guidance, code reviews, design discussions, and knowledge sharing.
Lead small projects or serve as the technical lead for data engineering initiatives when needed.
Effectively communicate technical challenges, risks, and progress with engineering teams, leadership, clients, and stakeholders.
Participate in technical interviews and contribute to hiring decisions.