ETL Developer

Vantor•McLean, VA

1d•$200,000 - $250,000•Onsite

About The Position

Vantor is seeking an experienced ETL Developer to provide onsite support for a geospatial visualization platform. The role develops, maintains, and optimizes data pipelines to ensure reliable data flow into a PostgreSQL database, working with ETL processes, data integration, large complex datasets, and geospatial data. The developer will learn and support the existing Apache NiFi system and its related AWS and PostgreSQL infrastructure, troubleshoot real-time issues, and build new NiFi flows following established best practices. The position works with analysts to understand data-layer needs and translate requirements into specific changes, while using a local containerized development environment, Git, and NiFi Registry. The role also includes database design and optimization, data transformation and analysis, API integration, and Python development for data science efforts or ETL prototypes. The position requires U.S. citizenship and an active TS/SCI security clearance with the required polygraph. The successful candidate must be willing to work onsite full-time.

Requirements

Bachelor’s degree in Computer Science, Systems Engineering, Geography, or a related technical field with at least 8 years of relevant experience; or a master’s degree with at least 7 years; or a doctorate with at least 6 years.
Active TS/SCI clearance with required polygraph.
U.S. citizenship.
Willingness to work onsite full-time.
Willingness to develop NiFi flows using existing best practices.
Strong understanding of ETL concepts and best practices.
Strong experience with databases for data management.
Proficiency in SQL for querying large datasets.
Experience with API integration and data retrieval.
Demonstrated experience with programmatic data manipulation.
Ability to rapidly develop proficiency with containerized development.
Ability to rapidly develop proficiency with AWS architecture.
Ability to rapidly develop proficiency with NiFi Registry and Git for code management.

Nice To Haves

Creativity to experiment with and recommend best-practice coding improvements.
Drive to adopt and recommend AI in development processes.
Ability to design and develop custom NiFi processors using Java.
Experience with AWS cloud services for data ingestion and processing.
Strong experience with PostgreSQL for data management.
Hands-on experience with Apache Airflow.
Hands-on experience with Apache NiFi.
Hands-on experience with NiFi Registry and Git.
Hands-on experience with Git, repository actions, and pull requests.
Hands-on experience with Python for data manipulation and API interfacing.
Experience working with geospatial data and related tools.
Understanding of basic data science concepts and tools.

Responsibilities

Design, develop, and manage ETL workflows using Apache NiFi for a geospatial visualization platform.
Learn the existing NiFi system and related AWS and PostgreSQL infrastructure to monitor and troubleshoot real-time issues.
Understand existing NiFi flows with minimal support and develop new flows using their best practices.
Incorporate feedback and requested changes into NiFi flows.
Take direction and communicate status through a ticketing system.
Work with analysts by email, chat, and in-person meetings to understand future data-flow needs and translate general requirements into specific actions.
Use a local development environment with containerized infrastructure, Git, and NiFi Registry.
Develop competency in geospatial database design and apply best practices for table creation, indexing, and unique constraints.
Incorporate analyst feedback about requirements for application data layers.
Programmatically manipulate and transform data from multiple sources.
Query and analyze large datasets using SQL.
Retrieve and process data from external systems via APIs.
Support data quality, integrity, and performance optimization.
Distinguish experimental prototypes from enterprise-level software requirements while working with Python and data science frameworks.
Develop Python in support of data science efforts or ETL prototypes.