Senior Data Specialist

McKesson•Mississauga, ON

3d•CA$99,100 - CA$132,100•Hybrid

About The Position

The Senior Data Specialist provides technical leadership in designing, developing, and optimizing cloud-native data solutions on the Azure Data platform, with a primary focus on Azure Databricks. This role is responsible for building enterprise-scale data pipelines, implementing modern lakehouse architectures, and enabling trusted, high-quality data products that support analytics, AI/ML, and business decision-making. The ideal candidate brings deep hands-on experience with Databricks, PySpark, advanced SQL, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Delta Lake, Delta Live Tables (DLT), and Change Data Capture (CDC) frameworks. This individual will play a key role in designing and supporting robust Medallion Architecture patterns while ensuring data quality, performance, security, and governance across the enterprise. Working closely with architects, product teams, analytics stakeholders, and governance teams, the Senior Data Specialist will deliver scalable, reusable, and production-ready data engineering solutions in a highly regulated environment.

Requirements

Expert-level hands-on Azure Databricks experience designing, building, and supporting enterprise-grade data solutions.
Strong expertise in PySpark and advanced SQL for large-scale data processing and optimization.
Extensive experience with: Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), Delta Lake, Delta Live Tables (DLT), Change Data Capture (CDC), Medallion Architecture
Deep understanding of Spark architecture, job optimization, partitioning strategies, and performance tuning.
Experience architecting and supporting end-to-end ETL/ELT pipelines in cloud-native environments.
Strong knowledge of data modeling, data warehousing, metadata management, and governance best practices.
Experience implementing CI/CD, source control, automated testing, and production deployment processes.
Ability to troubleshoot complex pipeline failures, data quality issues, performance bottlenecks, and integration challenges.
Bachelor's degree in Computer Science, Engineering, Information Systems, or related discipline; Master's degree preferred.
7+ years of experience in data engineering and enterprise data platform development.
3+ years of deep hands-on Azure Databricks engineering experience in production environments.
Proven track record building and supporting enterprise-scale data pipelines and Lakehouse implementations.
Experience working with large volumes of structured and semi-structured data.

Nice To Haves

Experience with Snowflake and modern cloud data ecosystems.
Experience with Kafka, streaming architectures, and event-driven data processing.
Exposure to AI/ML enablement and advanced analytics workloads within Databricks.
Azure certifications and Databricks certifications are highly preferred.
Healthcare, pharmaceutical, life sciences, or other regulated industry experience strongly preferred.
Master's degree

Responsibilities

Design, build, and optimize large-scale data pipelines using Azure Databricks, PySpark, and advanced SQL.
Develop and maintain batch and near real-time data ingestion frameworks using Azure Data Factory (ADF), Delta Lake, and CDC methodologies.
Build and support modern Lakehouse solutions leveraging Medallion Architecture (Bronze, Silver, Gold layers).
Design and implement Delta Live Tables (DLT) pipelines to improve reliability, maintainability, and data quality.
Develop highly scalable and resilient ETL/ELT frameworks that process high-volume enterprise datasets.
Optimize Spark workloads, partitioning strategies, job orchestration, and SQL performance for large-scale processing environments.
Implement robust monitoring, alerting, troubleshooting, and performance-tuning practices across data pipelines.
Establish and maintain data quality validation frameworks, lineage tracking, metadata management, and governance controls.
Implement automated testing and reconciliation processes to ensure data accuracy, completeness, and consistency.
Support regulatory, security, and compliance requirements through documented controls and audit-ready processes.
Collaborate with architects and stakeholders to translate business requirements into scalable technical solutions.
Drive engineering best practices, code reviews, CI/CD adoption, and reusable framework development.
Mentor junior engineers and provide technical leadership on Databricks and Azure-based implementations.
Evaluate emerging Azure and Databricks capabilities and recommend enhancements to improve platform performance and operational efficiency.