Data Engineer

Capgemini • Mississauga, ON

3d • Onsite

About The Position

We are seeking an experienced Data Engineer to join our team. In this role, you will be responsible for designing, building, and optimizing scalable data pipelines and ETL/ELT workflows. You will leverage Databricks (Unity Catalog, Delta Live Tables, Workflows, Asset Bundles), Apache Spark, and Scala to process large volumes of structured and unstructured data. You will also architect and manage data lakehouse solutions using Apache Iceberg and Delta Lake for table format management, versioning, time travel, and efficient data storage and retrieval at scale. Additionally, you will build and maintain robust backend data services and APIs, integrating Databricks with downstream systems to enable real-time and batch data consumption across enterprise platforms. Collaboration with data architects, analysts, and engineering teams to define data models, establish best practices, and drive continuous improvement across the data platform is a key aspect of this position.

Requirements

• 3-5 years of experience as a Data Engineer.
• Proficiency in Databricks (Unity Catalog, Delta Live Tables, Workflows, Asset Bundles).
• Experience with Apache Spark.
• Experience with Scala programming language.
• Experience with data lakehouse solutions.
• Experience with Apache Iceberg.
• Experience with Delta Lake.
• Experience building backend data services and APIs.
• Experience integrating Databricks with downstream systems.
• Experience with real-time and batch data consumption.
• Experience defining data models.
• Experience establishing best practices for data platforms.
• Experience driving continuous improvement across data platforms.

Responsibilities

• Design, build, and optimize scalable data pipelines and ETL/ELT workflows using Databricks (Unity Catalog, Delta Live Tables, Workflows, Asset Bundles), Apache Spark, and Scala to process large volumes of structured and unstructured data.
• Architect and manage data lakehouse solutions leveraging Apache Iceberg and Delta Lake for table format management, versioning, time travel, and efficient data storage and retrieval at scale.
• Build and maintain robust backend data services and APIs, integrating Databricks with downstream systems, enabling real-time and batch data consumption across enterprise platforms.
• Collaborate with data architects, analysts, and engineering teams to define data models, establish best practices, and drive continuous improvement across the data platform.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed