Senior Data Engineer

Labcorp•Durham, NC

7d•Onsite

About The Position

Labcorp is seeking a Senior Data Engineer to join our team in Durham, NC. This role involves designing and implementing end-to-end data engineering solutions using Databricks and Fabric tools, including data ingestion, transformation, and modeling. The engineer will develop and maintain scalable, reliable, and cost-optimized data pipelines using Spark, collaborate with internal teams and clients to understand business requirements and translate them into technical solutions, and conduct performance tuning and troubleshooting. The position also requires implementing data governance and security best practices, translating business requirements into technical documents, and working closely with architects, product managers, and the reporting team. The role operates in a fast-paced, agile development environment.

Requirements

Bachelor’s Degree
8 years of experience in designing and implementing data solutions
4 years of experience in data engineering
4 years experience with Databricks, Fabric, including a deep understanding of its architecture, data modeling, and real-time analytics.
6 years of experience in Spark, PySpark and Python.
4 years experience in SQL, Spark SQL, data modeling & RDBMS concepts.

Nice To Haves

Extensive experience with Databricks,Fabric, including a deep understanding of its architecture, data modeling, and real-time analytics.
Strong knowledge of Data Fabric services, particularly Data engineering, Data warehouse, Data factory, and Real- time intelligence.
Solid experience with Git, CI/CD, branching strategies.
Experience building ingestion frameworks for legacy systems.
Strong problem-solving skills, with ability to perform multi-tasking.
Familiarity with security best practices in cloud environments, Active Directory, encryption, and data privacy compliance.
Communicate effectively in both oral and written.
Experience in AGILE development, SCRUM and Application Lifecycle Management (ALM).
Preference given to current or former Labcorp employees.

Responsibilities

Design and implement end-to-end data engineering solutions by leveraging the full suite of Databricks, Fabric tools, including data ingestion, transformation, and modeling.
Design, develop and maintain end-to-end data pipelines by using spark, ensuring scalability, reliability, and cost optimized solutions.
Collaborate with internal teams and clients to understand business requirements and translate them into robust technical solutions.
Conduct performance tuning and troubleshooting to identify and resolve any issues.
Implement data governance and security best practices, including role-based access control, encryption, and auditing.
Translate Business requirements into high-quality technical documents including, data mapping, data processes, and operational support guides.
Work closely with architects, product managers and reporting team to collect functional and system requirements.
Work in fast-paced environment and perform effectively in an agile development environment.