Senior Data Engineer

Qode
Hybrid

About The Position

The Senior Data Engineer (Informatica & PySpark) position is a full-time role based in Cleveland, OH or Pittsburgh, PA, requiring 3 days onsite. This role demands 8-10 years of experience in Data Engineering and Data Analysis. Key responsibilities include ETL design, development, and optimization using Informatica PowerCenter/IDQ, and large-scale data processing, transformation, and analytics with PySpark. The engineer will also work with Hadoop technologies, Python, and Kafka for data pipelines, and apply strong understanding of database concepts, data design, data modeling, and ETL workflows. The position involves analyzing, designing, and coding ETL programs, project delivery using Agile methodology and Jira, and coordinating across the SDLC in client-facing capacities.

Requirements

  • 8–10 years of experience in Data Engineering and Data Analysis
  • Strong hands-on experience in Informatica PowerCenter/IDQ
  • Advanced skills in PySpark
  • Solid working knowledge of Hadoop technologies (HDFS, Hive, Sqoop, MapReduce)
  • Proficiency in Python and Kafka for streaming and batch data pipelines
  • Strong understanding of database concepts, data design, data modeling, and ETL workflows
  • Hands-on experience with Agile methodology and Jira
  • Proven ability in client-facing roles with strong communication and leadership skills

Nice To Haves

  • Exposure to AWS data components and analytics
  • Familiarity with machine learning models and AI concepts
  • Experience with data modeling tools such as Erwin

Responsibilities

  • Design, develop, and optimize ETL using Informatica PowerCenter/IDQ
  • Perform large-scale data processing, transformation, and analytics using PySpark
  • Analyze, design, and code ETL programs including data extraction, ingestion, quality checks, normalization, and loading
  • Deliver projects using Agile methodology and Jira
  • Coordinate across the SDLC in client-facing roles
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service