Software Developer - ETL - Senior

ITProFound Inc•Toronto, ON

8d•Onsite

About The Position

We have a job opening for a Software Developer - ETL - Senior for our direct client, Province of Ontario. The position is for the Health Services Cluster and is an onsite role with a duration of 1 year plus potential extensions.

Requirements

10+ years experience in designing and developing scalable Medallion Data Lakehouse architectures.
Expertise in data ingestion, transformation, and curation using Delta Lake and Databricks.
Experience integrating structured and unstructured data sources into star/snowflake schemas.
Experience building, automating, and optimizing complex ETL/ELT pipelines using Azure Data Factory (ADF), Databricks (PySpark, SQL, Delta Live Tables), and dbt.
Experience implementing orchestrated workflows and job scheduling in Azure environments.
Strong knowledge of relational (SQL Server, Synapse, PostgreSQL) and dimensional modeling.
Advanced SQL query optimization, indexing, partitioning, and data replication strategies.
Experience with Apache Spark, Delta Lake, and distributed computing frameworks in Azure Databricks.
Experience working with Parquet, ORC, and JSON formats for optimized storage and retrieval.
Deep expertise in Azure Data Lake Storage (ADLS), Azure Synapse Analytics, Azure SQL, Event Hubs, and Azure Functions.
Strong understanding of cloud security, RBAC, and data governance.
Proficiency in Python (PySpark), SQL, and PowerShell for data engineering workflows.
Experience with CI/CD automation (Azure DevOps, GitHub Actions) for data pipelines.
Experience implementing data lineage, cataloging, metadata management, and data quality frameworks.
Experience with Unity Catalog for managing permissions in Databricks environments.
Expertise in Power BI (DAX, data modeling, performance tuning).
Experience in integrating Power BI with Azure Synapse and Databricks SQL Warehouses.
Familiarity with MLflow, AutoML, and embedding AI-driven insights into data pipelines.

Responsibilities

Designing and developing scalable Medallion Data Lakehouse architectures.
Data ingestion, transformation, and curation using Delta Lake and Databricks.
Integrating structured and unstructured data sources into star/snowflake schemas.
Building, automating, and optimizing complex ETL/ELT pipelines using Azure Data Factory (ADF), Databricks (PySpark, SQL, Delta Live Tables), and dbt.
Implementing orchestrated workflows and job scheduling in Azure environments.
Advanced SQL query optimization, indexing, partitioning, and data replication strategies.
Working with Parquet, ORC, and JSON formats for optimized storage and retrieval.
Implementing data lineage, cataloging, metadata management, and data quality frameworks.
Integrating Power BI with Azure Synapse and Databricks SQL Warehouses.
Embedding AI-driven insights into data pipelines.