Senior Data Engineer

Onzze

13d

About The Position

We are looking for a Senior Data Engineer to join a global automotive industry client program. The individual will be co-responsible for the data platform, working in partnership with an Analytics Engineer and dividing responsibilities by modules. The main objective will be to build and operate the medallion architecture-based pipeline — from data ingestion in Azure, through the Bronze and Silver layers, to a well-structured Gold layer — ensuring governance, quality, and CI/CD discipline. The idea is to enable the Analytics Engineer to use this Gold layer to generate insights, BI, and assets prepared for Data Science, without needing to fix structural issues in the data pipeline.

Requirements

• Relevant experience in Data Engineering, with at least 3 years of hands-on experience with Databricks and production medallion architecture.
• Knowledge of Python, SQL, and PySpark.
• Previous experience with Azure Data Lake Storage Gen2, Unity Catalog, Azure DevOps, and Databricks Workflows.
• Fluent English is mandatory, as the position will involve direct collaboration with a global program team.
• Proactivity: Anticipate problems and propose improvements before being asked, including for issues identified outside one's own module.
• Direct Communication: Provide proactive status updates — what has been completed, what is next, and what is blocked — signal risks and delays before being questioned, and document important decisions in writing on the same day.
• Ownership: Autonomously manage one's area and be the primary point of contact for decisions within one's scope, without constant supervision.
• Genuine Care: Demonstrate real concern for the outcome, not just for task execution. This means raising a concern that no one requested, not simply spending more hours online.

Nice To Haves

• Previous experience integrating Salesforce as a data source.
• Databricks certification, such as Databricks Certified Data Engineer Professional or Associate.
• Experience in the automotive sector or in other regulated manufacturing programs with multiple plants.

Responsibilities

• Pipeline Architecture: Design and implement end-to-end medallion architecture in Databricks — ingestion into Azure Data Lake Storage Gen2 and construction of Bronze, Silver, and Gold layers — for batch and near real-time sources.
• Data Ingestion: Ingest data from the program's source systems into Azure, handling schema evolution, validation, and reprocessing in an environment with multiple source systems operating simultaneously.
• Governance: Utilize Unity Catalog to ensure access control, lineage, and data governance across all layers and workspaces.
• Platform and CI/CD: Build and maintain CI/CD pipelines in Azure DevOps and orchestrate production pipelines using Databricks Workflows.
• Ownership Model: Share platform responsibility with the Analytics Engineer, clearly documenting who is responsible for what and within what deadlines. Data Engineer: ingestion, transformation logic, and structure/quality of the Gold layer. Analytics Engineer: generation of insights, enablement for Data Science, delivery of BI, and documentation built on top of this layer.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed