Data Engineer SME

Dark Wolf Solutions•Chantilly, VA

130d•$170,000 - $210,000•Onsite

About The Position

Dark Wolf Solutions is seeking a highly motivated and experienced Data Engineer to assist with strategic planning and oversee implementation of the cloud-based data environment. The work includes engaging regularly with data scientists, analysts, and managers. This role is located in Chantilly, VA.

Requirements

Effectively facilitates communication and collaboration as a technical liaison between system engineers, data engineers, data scientists, analysts, and non-technical managers/personnel.
Proficient in utilizing AWS cloud services, including long-term storage solutions and cloud-based database services like Databricks and Elastic MapReduce (EMR).
Experienced in designing and implementing SQL database structures and mappings between databases.
Knowledgeable in network-attached storage (NAS) systems and their implementation.
Skilled in creating and maintaining automated deployment scripts for streamlined software releases.
Expert in managing and executing large-scale data migration projects, ensuring data integrity and minimal disruption.
Hands-on experience with large-scale data compute and processing clusters such as Hadoop, optimizing performance and scalability.
Adept at test-driven development within a secure on-premise cluster, selecting and employing the most efficient languages for the task, including Apache NiFi, Java, Python, and SQL.
In-depth understanding of database architecture and performance design methodologies, providing system-tuning recommendations for technologies like Hadoop Hive, Apache NiFi, and Impala.
Continuously improves and maintains the ETL process through the implementation and standardization of data flows using Apache NiFi and other ETL tools.
US Citizen with an active Top Secret/Sensitive Compartmented Information (TS/SCI) security clearance with polygraph.

Nice To Haves

Possesses in-depth knowledge of the data environment and on-premises compute infrastructure.
Proficient in storage and backup recovery systems, ensuring data availability and integrity.
Experienced with Data Quality and Data Governance concepts, implementing best practices for data management.
Skilled in System Administration and Linux Administration, maintaining system stability and performance.
Adept at transforming data using High Capacity Compute (HCC) techniques to derive valuable insights.
Experienced in administering system rights and responsibilities for the on-premises cluster and cloud infrastructure, ensuring secure access and resource allocation.

Responsibilities

Providing comprehensive support to analysts by delivering large datasets, methodologies, and impactful data visualizations to address critical intelligence needs.
Aiding in the development and maintenance of a robust cloud-based data environment for efficient data transport, storage, ETL processes, and solution dissemination.
Assisting in data engineering, cloud architecture design, and application development efforts, contributing to the overall success of projects.
Engaging regularly with data scientists, analysts, and managers.
Assisting with strategic planning and oversee implementation of the cloud-based data environment, to include mapping of data sources and access controls.
Developing code, data models, and documentation to standards; providing systems administration and programming support for ETL processes and data infrastructure efforts; and training and conducting knowledge transfer to team members on issues and technologies related to the ETL process, on premise high capacity compute cluster, and administrative duties.
Coordinating with external data and platform providers to ensure the smooth functioning of the systems and data flows, and to accomplish any needed changes and coordinate with experts to assist with technical aspects required to acquire new datasets or data management technologies for inclusion in the environment.
Supporting the cross-domain transfer and integration of data.
Assisting with strategic planning and oversee implementation of the cloud-based data environment, to include mapping of data sources and access controls.
Developing code, data models, and documentation to standards.
Providing systems administration and programming support for ETL processes and data infrastructure efforts.
Training and conducting knowledge transfer to team members on issues and technologies related to the ETL process, on premise high capacity compute cluster, and administrative duties.
Coordinating with external data and platform providers to ensure the smooth functioning of the systems and data flows, and to accomplish any needed changes.
Coordinating with experts to assist with technical aspects required to acquire new datasets or data management technologies for inclusion in the environment.
Supporting the cross-domain transfer and integration of data, to include using on-premises cluster, cloud environment, and SQL-based systems such as PostgreSQL and Impala.