ETL Developer

Vantor•McLean, VA

1d•Onsite

About The Position

Vantor is seeking an experienced ETL Developer to provide full-time onsite support for a geospatial visualization platform. The role centers on developing, maintaining, and optimizing data pipelines to ensure reliable data flow into a PostgreSQL database, including integration and processing of large, complex datasets and geospatial data. The developer will learn and support the existing Apache NiFi system and its related AWS and PostgreSQL infrastructure, create and revise NiFi flows using established practices, and work with analysts to translate data needs into implementation tasks. The position also involves programmatically manipulating and transforming data, querying large datasets with SQL, retrieving data through APIs, supporting data quality and performance, and contributing Python work for data science efforts or ETL prototypes. This role requires U.S. citizenship and an active TS/SCI clearance with the required polygraph.

Requirements

Bachelor’s degree and 8 or more years of relevant experience, or a master’s degree and 7 or more years of experience, or a doctorate and 6 or more years of experience.
Degree in Computer Science, Systems Engineering, Geography, or a related technical field.
Active TS/SCI clearance with required polygraph.
U.S. citizenship.
Willingness to work onsite full-time.
Willingness to develop NiFi flows using existing best practices.
Strong understanding of ETL concepts and best practices.
Strong experience with databases for data management.
Proficiency in SQL for querying large datasets.
Experience with API integration and data retrieval.
Demonstrated experience with programmatic data manipulation.
Ability to rapidly develop proficiency with containerized development.
Ability to rapidly develop proficiency with AWS architecture.
Ability to rapidly develop proficiency with NiFi Registry and Git for code management.

Nice To Haves

Creativity to experiment with and recommend best-practice coding improvements.
Drive to adopt and recommend AI in development processes.
Ability to design and develop custom NiFi processors using Java.
Experience with AWS cloud services for data ingestion and processing.
Strong experience with PostgreSQL for data management.
Hands-on experience with Apache Airflow.
Hands-on experience with Apache NiFi.
Hands-on experience with NiFi Registry and Git.
Hands-on experience with Git, repository actions, and pull requests.
Hands-on experience using Python for data manipulation and API interfacing.
Experience working with geospatial data and related tools.
Understanding of basic data science concepts and tools.

Responsibilities

Design, develop, and manage Apache NiFi ETL workflows for a geospatial visualization platform.
Learn the existing NiFi system and related AWS and PostgreSQL infrastructure to monitor and troubleshoot real-time issues.
Understand existing NiFi flows with minimal support and develop new flows following their best practices.
Incorporate feedback and make requested changes to NiFi flows.
Take direction and communicate status through a ticketing system.
Work with analysts by email, chat, and in person to understand future data flow needs and turn general requirements into specific actions.
Use a local development environment with containerized infrastructure and version control through Git and NiFi Registry.
Develop competency in geospatial database design and apply best practices for table creation, indexing, and unique constraints.
Incorporate analyst feedback on requirements for application data layers.
Programmatically manipulate and transform data from multiple sources.
Query and analyze large datasets using SQL.
Retrieve and process data from external systems through APIs.
Support data quality, integrity, and performance optimization efforts.
Develop Python in support of data science efforts or ETL prototypes.