Senior Data Engineer

Elliott Group, Ebara Corp•Jeannette, PA

About The Position

The Data Engineer builds robust, complex data pipelines to move data from various source systems into the Enterprise Data Lakehouse. The role architects, troubleshoots, and designs complex workflows and transformations. It involves developing and documenting ETL pipelines, standardizing data integration, monitoring data flows across databases and cloud and on-premise environments, and improving data quality and lineage. The engineer also develops Master Data Management processes, data-cleansing algorithms, and reliable logging and troubleshooting methods. The position works with business users across Operations, Sales, Marketing, and Finance in global locations, as well as visualization engineers, data analysts, IT, and data scientists, to bring new data sources into the pipeline. It requires independent technical problem-solving, consultative advice, initiative, and the ability to understand and influence complex business concepts and resolve challenging application and data integration issues.

Requirements

Bachelor's degree in Computer Science, Information Systems, Data Science, or Analytics.
At least 10 years of hands-on experience as an ETL engineer.
At least 3 years of Databricks experience.
Experience with database systems such as Oracle and SQL Server.
Experience with cloud technologies, preferably AWS.
Experience with SQL, data modeling, and Informatica, Fivetran, or Azure Data Factory.
Deep understanding of extracting data from ODBC sources and using APIs.
Basic Python skills.
Ability to work with multiple applications and business units, and collaborate with data analysts, visualization engineers, and data scientists.
Sophisticated skills in understanding, influencing, driving, and negotiating with internal customers.
Initiative and extensive knowledge to resolve complex issues and develop new or novel solutions.

Responsibilities

Review source-to-target mapping documents.
Design and document ETL pipelines using industry-standard ETL tools.
Streamline and build robust data pipelines with clear data lineage.
Monitor data flows across databases, cloud environments, and on-premise systems.
Implement best practices and standardize data integration across sources.
Profile and detect data anomalies and provide solutions to improve data quality.
Build and implement Master Data Management processes, including defining algorithms and writing code to cleanse data.
Establish robust and efficient data flows with logging and effective troubleshooting methods.
Work in a fast-paced and changing environment.
Identify, understand, and analyze complex business concepts.
Work independently and provide consultative advice on technical issues.
Develop novel resolutions to complex issues and work on application and data integration issues.
Interact regularly with internal business users across Operations, Sales, Marketing, and Finance, as well as visualization engineers, data analysts, IT, and data scientists, to enable new data sources in the pipeline.