Senior IT Data Engineer
About The Position
Diverse Agile Solutions (DAS) is seeking a highly skilled Senior IT Data Engineer to support the design, development, and optimization of enterprise data platforms within a federal environment. This role is critical to ensuring data integrity, accessibility, and performance across complex systems, enabling data-driven decision-making at scale. The ideal candidate will have deep experience in building and maintaining modern data pipelines, working with distributed data platforms, and supporting data governance and quality initiatives. As a Senior IT Data Engineer, you will play a key role in the planning and execution of data management strategies, ensuring the acquisition, control, protection, and enhancement of critical data assets. You will work with diverse data sources, transforming raw data into reliable, high-quality datasets that support analytics, reporting, and operational needs. You will also be responsible for designing and maintaining scalable data pipelines and ensuring seamless data flow across systems while adhering to best practices in data engineering and DevOps.
Requirements
- 5+ years of experience in application or data development (Python preferred)
- 5+ years of experience with data integration and ingestion tools (e.g. Apache NiFi)
- Strong experience building and managing data pipelines in Cloudera Data Platform (CDP)
- Proficiency in Python and SQL , including tools such as PySpark, pandas, and dbt
- Advanced knowledge of distributed data and computing platforms, including: Apache NiFi Hadoop ecosystem (MapReduce, Hive, HBase) Kafka Apache Spark
- Experience with Microsoft SQL Server and relational databases
- Familiarity with Java for data processing or integration tasks
- Experience with UNIX/Linux environments , including shell scripting
- Hands-on experience implementing and maintaining CI/CD pipelines
- Strong understanding of data architecture, data modeling, and performance tuning
Nice To Haves
- Experience working in federal or regulated environments
- Familiarity with data governance frameworks and best practices
- Experience with cloud-based data platforms (AWS, Azure, or similar)
- Strong problem-solving skills and ability to work independently in fast-paced environments
Responsibilities
- Design, develop, and maintain robust, scalable data pipelines to ingest, process, and transform data from multiple sources
- Ensure data quality, integrity, and consistency across enterprise data platforms
- Facilitate data acquisition and integration from structured and unstructured sources
- Build and optimize ETL/ELT processes to support analytics and reporting needs
- Monitor and troubleshoot data pipelines and workflows to ensure high availability and performance
- Implement and maintain CI/CD pipelines for data engineering workflows
- Collaborate with cross-functional teams including developers, analysts, and stakeholders to deliver data solutions
- Support data governance, security, and compliance initiatives
- Participate in Agile development processes, including Scrum and Kanban ceremonies
Benefits
- Work on high-visibility, mission-critical systems
- Collaborate with top-tier engineering and data professionals
- Opportunity for contract extension and long-term engagement
- Competitive compensation and professional growth opportunities
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
Number of Employees
1-10 employees
