Senior IT Big Data Developer

Diverse Agile Solutions•Washington, DC

1d•$70 - $82•Hybrid

About The Position

Diverse Agile Solutions (DAS) is seeking a highly skilled Senior IT Big Data Developer to support the Federal Reserve Board’s Data Architecture, Technology, and Analytics (DATA) Section within the Division of Research & Statistics (R&S). This role offers an opportunity to help transform how critical economic and financial data is ingested, managed, analyzed, and delivered to support research and policy initiatives. The Senior IT Big Data Developer will serve as a key contributor in designing, developing, and optimizing enterprise-scale data architectures and data engineering solutions. This individual will be responsible for building and maintaining scalable data pipelines, modernizing data platforms, integrating diverse data sources, and supporting advanced analytics initiatives. The ideal candidate is a hands-on data engineering professional with deep expertise in database design, large-scale data processing, cloud technologies, ETL/ELT development, and enterprise data architecture. The successful candidate will collaborate with economists, researchers, data scientists, and technical teams to deliver high-performance data solutions that support mission-critical research and policy decisions.

Requirements

Bachelor’s degree in Computer Science, Information Technology, Engineering, Data Science, or a related technical discipline.
Minimum of 7 years of experience in data engineering, big data development, data architecture, or related technical roles.
Experience designing and implementing enterprise data platforms supporting large-scale analytical workloads.
Proven experience developing and optimizing enterprise data pipelines and integration frameworks.
Experience supporting complex research, financial, or analytical environments.
Advanced SQL expertise.
Strong experience with relational database platforms: PostgreSQL, Microsoft SQL Server, MySQL.
Database design, administration, and performance optimization.
Data modeling and enterprise information architecture.
Advanced proficiency with Python, R.
Experience with additional programming languages such as Java, Scala, JavaScript, Perl.
Experience working with distributed computing environments, scalable data processing frameworks, large-scale data storage architectures, and high-volume data workloads.
Design and automation of ETL/ELT workflows.
Data integration and transformation solutions.
Workflow orchestration tools including Apache Airflow, Prefect, Dagster, AWS Step Functions.
Experience with Amazon Web Services (AWS), Microsoft Azure, Snowflake.
Experience migrating data pipelines between cloud and on-premises environments.
CI/CD pipeline implementation and automation.
DataOps platform development and support.
GitLab and GitHub source control management.
Linux-based development environments.
U.S. Citizen Required
Ability to pass a Federal Background Investigation

Nice To Haves

Advanced degree (Master’s or Ph.D.).
Experience working with economic, financial, or research-related data environments.
Understanding of time-series data structures and forecasting methodologies.
Experience with NoSQL databases and graph database technologies.
Experience developing, training, deploying, and maintaining machine learning models.
Experience implementing enterprise data warehouses utilizing CDC methodologies.
Knowledge of enterprise architecture frameworks and modern data governance practices.
Experience supporting highly regulated or federal government environments.
Strong analytical and critical thinking abilities.
Exceptional problem-solving and troubleshooting skills.
Excellent verbal and written communication skills.
Ability to communicate technical concepts to both technical and non-technical stakeholders.
Strong customer-service mindset and collaborative approach.
Self-motivated and capable of managing multiple priorities simultaneously.
Ability to work independently while contributing effectively within cross-functional teams.

Responsibilities

Design, develop, and maintain enterprise-scale data architectures, data warehouses, and data lake solutions.
Build and optimize scalable ETL/ELT pipelines for ingesting, processing, and transforming structured and unstructured data.
Develop and maintain high-performance data integration workflows supporting economic research and analytics initiatives.
Design conceptual, logical, and physical data models aligned with enterprise architecture standards.
Optimize data storage, processing, and delivery frameworks to support large-scale analytical workloads.
Implement workflow orchestration and automation solutions using tools such as Apache Airflow, Prefect, Dagster, or AWS Step Functions.
Develop solutions that support data migration between on-premises and cloud environments.
Collaborate with researchers, economists, and business stakeholders to understand data requirements and deliver scalable solutions.
Perform root cause analysis and troubleshoot complex data quality, integration, and performance issues.
Design and implement Change Data Capture (CDC) processes to support enterprise data warehouse solutions.
Develop, test, deploy, and maintain data applications and services using industry best practices.
Implement DataOps and CI/CD processes to improve deployment automation and operational efficiency.
Support machine learning and advanced analytics initiatives through the development of reliable data pipelines and feature engineering frameworks.
Ensure data governance, security, reliability, and scalability standards are met across enterprise platforms.
Document architecture designs, technical specifications, and operational procedures.