Senior Analyst (Data Engineer)

Office of the Superintendent of Financial Institutions Canada•Toronto, ON

3d•CA$105,200 - CA$135,300•Hybrid

About The Position

The Data Engineer will play a key role within the Data Engineering team, supporting the delivery of high-quality, reliable, and well-governed datasets. This role focuses on ensuring timely, accurate, and complete data through strong data engineering practices, data quality validation, and analytical insight. The Data Engineer will contribute to building scalable pipelines, improving data quality processes, and enhancing the overall data ecosystem through continuous improvement and innovation.

Requirements

Recent (within the last three years) and significant (five years) experience developing, maintaining, and optimizing data pipelines in Azure Synapse or Databricks to support high-quality data ingestion, transformation, and validation.
Recent (within the last three years) experience performing data profiling, cleansing, and standardization to identify anomalies, inconsistencies, and data quality issues across large datasets.
Recent (within the last three years) experience working with cloud data platforms (e.g., Azure Synapse, Databricks, Fabric, or similar) including data lake concepts, privacy, and secure handling of sensitive data.
Recent (within the last three years) experience using Python and Spark to build scalable data transformations, automation scripts, and data quality routines.
Recent (within the last three years) experience with DevOps Practices including Git repository management, CI/CD pipelines, and automated deployment of data engineering artifacts.
Demonstrated experience applying automated testing approaches for data pipelines, including unit tests, data quality checks, and regression validation.
Demonstrated experience integrating data from various source systems like SFTP, APIs, etc., and other structured or semi-structured sources.
Experience working with banking or financial services data such as regulatory, risk, compliance, payments, or reporting datasets across typical industry domains.
Ability to communicate effectively in writing.
Ability to communicate effectively verbally.
Knowledge of diverse data source systems and methods for extracting data from platforms such as SFTP, APIs, databases, and file-based sources.
Knowledge of designing and building data pipelines, including orchestration, transformation, and end-to-end workflow management.
Knowledge of framework for Data ingestion and Data Quality covering extraction, validation, monitoring, and standardizing processes.
Knowledge of working with structured, semi-structured, and unstructured datasets.
Knowledge of data engineering best practices for designing and maintaining optimized, reliable, and scalable data pipelines.
Collaboration competency.
Innovation competency.
Critical Thinking competency.
Results Orientation competency.

Nice To Haves

A relevant recognized professional designation.
Recent (within the last three years) experience designing and implementing data lake architectures, including medallion (Bronze/Silver/Gold) patterns and secure handling of sensitive datasets.
Experience supporting data migration initiatives, including schema mapping, reconciliation, and validation across multiple environments.
Experience implementing data validation frameworks, automated testing approaches, or data quality monitoring processes within data pipelines.
Knowledge on integrating data from APIs or other structured/semi-structured sources into enterprise data environments.
Ability to quickly learn new tools and techniques, including those from open-source software.

Responsibilities

Develop, maintain, and optimize data pipelines in Azure Synapse / Microsoft Fabric to support high-quality data ingestion, transformation, and validation.
Extract and integrate data from APIs and other structured or semi-structured sources as part of automated data quality workflows.
Perform data profiling, cleansing, and standardization to identify anomalies, inconsistencies, and data quality issues across large datasets.
Apply data lake best practices, including privacy and security controls such as masking, anonymization, and secure handling of sensitive data.
Support data migration activities by validating data completeness, accuracy, and consistency across environments and systems.
Use Python and Spark to build scalable data transformations, automation scripts, and data quality routines.
Leverage Azure Logic Apps and Azure Functions to automate workflows and support event-driven data processing.
Use DevOps for Git repository management, CI/CD pipelines, YAML-based definitions, and automated deployment of data engineering artifacts.
Implement environment-specific configuration management using variables, parameter files, and secure key handling.
Apply automated testing approaches for data pipelines, including unit tests, data quality checks, and regression validation.
Design solutions that support auditability, traceability, and data retention requirements.