Engineer
About The Position
Must Have Technical/Functional Skills Design and Implementation: o Creating and implementing data flow solutions using NiFi, including defining data sources, destinations, and transformation processes. Ingestion and Transformation: o Managing data ingestion from various sources (e.g., files, databases, APIs) and transforming data into desired formats for downstream consumption. Routing and Enrichment: o Configuring NiFi to route data to appropriate destinations and enriching data with additional information (e.g., timestamps, location). Scalability and Performance: o Ensuring data flows are scalable and performant to handle varying data volumes and velocity. NiFi Cluster Management: o Maintaining and improving NiFi clusters, including configuration, optimization, and troubleshooting. Microservices Integration: o Developing and integrating microservices to support data conditioning, format validation, and transformation processes. Troubleshooting and Issue Resolution: o Identifying and resolving issues related to data flows, NiFi configurations, and data integration processes
Requirements
- Strong NiFi Expertise: Deep understanding of NiFi architecture, processors, controllers, and data flow management capabilities.
- Data Integration and ETL: Experience with data integration, ETL processes, and data modeling concepts.
- Programming Languages: Proficiency in Java, Python, or other relevant scripting languages.
- Database and SQL: Familiarity with databases, SQL, and data manipulation techniques.
- Cloud Technologies: Experience with cloud platforms (e.g., AWS, Azure, GCP) and their integration with NiFi.
- Big Data Technologies: Knowledge of big data technologies like Kafka.
Responsibilities
- Creating and implementing data flow solutions using NiFi, including defining data sources, destinations, and transformation processes.
- Managing data ingestion from various sources (e.g., files, databases, APIs) and transforming data into desired formats for downstream consumption.
- Configuring NiFi to route data to appropriate destinations and enriching data with additional information (e.g., timestamps, location).
- Ensuring data flows are scalable and performant to handle varying data volumes and velocity.
- Maintaining and improving NiFi clusters, including configuration, optimization, and troubleshooting.
- Developing and integrating microservices to support data conditioning, format validation, and transformation processes.
- Identifying and resolving issues related to data flows, NiFi configurations, and data integration processes
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Industry
Professional, Scientific, and Technical Services
Education Level
No Education Listed
Number of Employees
5,001-10,000 employees
