ETL Developer

Ideal Innovations, Inc. • Bethesda, MD

1d • $87,100 - $133,833 • Onsite

About The Position

The ETL Developer provides technical support for the design, development, implementation, and testing of customer tools and applications that extract, transform, and load data into an enterprise Data Lake. Working within a DevOps framework, the role contributes to major projects throughout the software development lifecycle and follows architectural best practices to deliver robust, scalable, secure data-flow solutions for critical customer needs.

The position centers on designing, implementing, and optimizing enterprise-scale Apache NiFi data pipelines for large-scale ingestion, transformation, and processing from diverse sources. The developer supports both batch and real-time processing, tunes performance through processor configuration, memory management, and load balancing, and troubleshoots data-flow issues, performance bottlenecks, data discrepancies, and integration failures. The role also integrates NiFi with databases, big-data ecosystems, and cloud platforms, and documents data flows, mappings, architectures, and operating procedures. Collaboration with data architects, data engineers, application and service developers, and other stakeholders is essential.

This is an onsite position in Bethesda, Maryland, with a Monday–Friday, 40-hour work week. Applicants must be U.S. citizens and hold an active TS/SCI clearance with a full-scope polygraph.

Requirements

• Active TS/SCI security clearance with a full-scope polygraph; U.S. citizenship is required.
• Bachelor’s degree with at least 4 years of relevant experience, or a master’s degree with at least 2 years of relevant experience; experience may be considered in lieu of a degree.
• Direct experience designing, developing, and managing complex NiFi data-flow solutions in large-scale enterprise environments.
• Experience with programming languages such as Java and/or Python, and scripting for automation.
• Experience writing and optimizing complex queries and interacting with relational and NoSQL databases such as PostgreSQL, Elasticsearch, and DynamoDB.
• Experience with real-time streaming and REST API integration.
• Experience with cloud platforms such as AWS, Azure, or OCI and related data services.
• Ability to analyze complex data challenges, identify root causes, and implement effective solutions.
• Ability to collaborate with cross-functional teams and clearly articulate technical concepts.

Nice To Haves

• Experience deploying ETL solutions in an AWS environment.
• Experience developing custom NiFi processors with Java and/or Python.
• Knowledge of NiFi architecture, processors, and configurations, with hands-on experience using NiFi Registry and NiFi clustering for high availability and scalability.

Responsibilities

• Design and implement enterprise-level Apache NiFi ETL pipelines for large-scale data ingestion, transformation, and processing from diverse sources.
• Optimize NiFi data flows for batch and real-time processing, including processor tuning, memory management, and load balancing.
• Identify, diagnose, and resolve complex NiFi data-flow issues, performance bottlenecks, data discrepancies, and integration failures.
• Integrate NiFi with databases, big-data ecosystems, and cloud platforms and services, including AWS, OCI, Azure, Kafka, Elasticsearch, S3, and SQS/SNS.
• Follow established NiFi development, deployment, security, and governance practices and enterprise data-management policies.
• Create and maintain documentation for NiFi data flows, mappings, architectures, and standard operating procedures, supporting knowledge transfer.
• Collaborate with data architects, data engineers, application and service developers, and other stakeholders to translate business requirements into technical solutions.
• Support the design, development, implementation, and testing of customer tools and applications throughout the software development lifecycle in a DevOps framework.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level