Sr. IT Big Data Developer

Diverse Agile Solutions•Washington, DC

8h•Onsite

About The Position

Diverse Agile Solutions is seeking a highly skilled Senior IT Big Data Developer (Data Architect/Engineer) to support the Federal Reserve Board's Data Architecture, Technology, and Analytics (DATA) organization within the Division of Research & Statistics. The DATA team is responsible for transforming how economic research data is collected, managed, integrated, analyzed, and visualized across the organization. This position is ideal for an experienced Data Architect and Data Engineer who enjoys designing scalable enterprise data platforms, building modern data pipelines, optimizing large-scale data processing systems, and supporting data-driven research initiatives. The successful candidate will collaborate closely with economists, data scientists, researchers, and software engineers to design next-generation data architecture supporting critical economic policy research.

Requirements

Bachelor's degree in Computer Science, Information Technology, Engineering, or a related technical discipline.
Minimum 7 years of professional experience in Data Engineering, Data Architecture, or Big Data Development.
Advanced experience developing complex SQL solutions.
Extensive experience with relational databases including: PostgreSQL Microsoft SQL Server MySQL
Advanced programming experience with: Python R
Experience designing scalable enterprise data platforms.
Experience building automated ETL/ELT pipelines.
Experience developing scalable data processing solutions.
Experience with GitLab and/or GitHub source control.
Experience working in Linux environments.
Strong software engineering background including development, testing, and deployment.
Excellent written and verbal communication skills.
Strong analytical and troubleshooting abilities.
U.S. Citizenship Required

Nice To Haves

Experience with one or more of the following technologies is highly desirable:
Data Engineering Apache Airflow Prefect Dagster AWS Step Functions
Cloud Platforms Amazon Web Services (AWS) Microsoft Azure Snowflake
Enterprise Data Technologies Enterprise Data Lakes Data Warehouses CDC (Change Data Capture) DataOps CI/CD Pipelines
Big Data Distributed Computing Large Scale Data Processing High-Volume Data Optimization
Additional Technologies NoSQL Databases Graph Databases
Machine Learning
Java Scala JavaScript Perl
Domain Experience Time Series Data Forecasting Analytics Economic Research Financial Data Research Organizations
Technical Skills Programming Languages Python R SQL Java (Preferred) Scala (Preferred) JavaScript (Preferred) Perl (Preferred)
Databases PostgreSQL Microsoft SQL Server MySQL NoSQL Graph Databases
Cloud AWS Microsoft Azure Snowflake
Data Engineering ETL / ELT Apache Airflow Prefect Dagster AWS Step Functions
Data Lakes Data Warehouses Change Data Capture (CDC) DataOps CI/CD
Tools GitLab GitHub Linux

Responsibilities

Design and implement scalable enterprise data architectures supporting research and analytics initiatives.
Develop, optimize, and maintain enterprise data pipelines and automated ETL/ELT workflows.
Design conceptual, logical, and physical data models.
Build highly scalable data processing frameworks capable of supporting large datasets.
Develop data integration solutions for structured and unstructured data sources.
Optimize database performance across enterprise environments.
Design enterprise data lake and data warehouse solutions.
Implement Change Data Capture (CDC) methodologies.
Develop workflow orchestration using tools such as Apache Airflow, Prefect, Dagster, or AWS Step Functions.
Support migration of data platforms between on-premises and cloud environments.
Perform root cause analysis of data quality and business process issues.
Collaborate with economists and technical teams to deliver trusted, high-quality data assets.
Develop software solutions supporting enterprise data services.
Participate in application testing, deployment, and production support.
Build and maintain CI/CD pipelines supporting DataOps initiatives.
Document architecture, standards, and technical designs.
Recommend improvements to enterprise data strategy and governance.