Senior Data Engineer

NVI • Washington, DC

23d • Onsite

About The Position

Designs, develops, and supports enterprise data integration and analytics environments supporting operational reporting and mission analytics. Create and maintain optimal data pipeline architecture, including optimal extraction, transformation and loading of data, leveraging Microsoft Fabric notebooks, Apache Spark (SQL, Python, Scala) and other Microsoft Azure or open-source tools as required. Maintain high quality documentation about data structures in shared data governance tools for the business to access. Collaborate with data owners, data stewards, analysts, software developers, and Database Administrators (DBAs). Experience with Apache Spark, Python, SQL or relational databases, Hadoop, Databricks and data pipeline management in Spark for batch and streaming. Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.

Requirements

• Must be a US citizen
• Must Reside in the DMV Area (DC, Maryland, Virginia)
• Must have a minimum of 10-years Experience
• Experience with Apache Spark, Python, SQL or relational databases, Hadoop, Databricks and data pipeline management in Spark for batch and streaming.

Nice To Haves

• Master's Degree Highly Preferred
• Power Platform certifications preferred.
• Relevant federal IT experience, technical capability, and operational support experience aligned to assigned labor category.

Responsibilities

• Designs, develops, and supports enterprise data integration and analytics environments supporting operational reporting and mission analytics.
• Create and maintain optimal data pipeline architecture, including optimal extraction, transformation and loading of data, leveraging Microsoft Fabric notebooks, Apache Spark (SQL, Python, Scala) and other Microsoft Azure or open-source tools as required.
• Maintain high quality documentation about data structures in shared data governance tools for the business to access.
• Collaborate with data owners, data stewards, analysts, software developers, and Database Administrators (DBAs).
• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior