Data Engineer

Scotiabank • Toronto, ON

13d • Onsite

About The Position

The Data Engineer will play a critical role in designing, building, and supporting scalable, secure, and resilient data solutions across enterprise cloud data platforms. This role will focus on modern data engineering practices, including Azure, Databricks, Unity Catalog, ETL/ELT pipeline development, dbt-based transformation, CI/CD automation, and data platform modernisation. The successful candidate will work closely with business stakeholders, product teams, data architecture, platform engineering, and application teams to deliver reliable data pipelines and high-quality data products that support reporting, analytics, operational decision-making, and enterprise data initiatives.

Requirements

• 4+ years of experience working with data warehouses, data lakes, lakehouse platforms, or enterprise data platforms.
• 4+ years of experience designing, developing, and supporting ETL/ELT data pipelines.
• Strong experience working with structured, semi-structured, and unstructured data.
• Hands-on experience with data ingestion, transformation, validation, reconciliation, and distribution patterns.
• Strong understanding of data modelling, SQL development, performance tuning, and pipeline optimisation.
• Experience building resilient, scalable, and maintainable data engineering solutions for enterprise environments.
• Strong hands-on experience with Azure cloud data services, including Azure Data Lake Storage, Azure Data Factory, and related cloud-native data platform components.
• Strong experience with Databricks, Spark, Delta Lake, and lakehouse architecture.
• Practical experience with Unity Catalog for data governance, access control, metadata management, and secure data sharing.
• Experience with Databricks Auto Loader, workflow orchestration, notebook development, job scheduling, and production-grade pipeline deployment.
• Understanding of cloud security, access management, data protection, and enterprise governance standards.
• Hands-on experience with dbt for data transformation, modular SQL development, testing, documentation, and deployment.
• Strong understanding of ELT design patterns, incremental models, reusable transformation logic, and data quality checks.
• Ability to design transformation layers that are maintainable, auditable, and aligned with enterprise data standards.
• Strong SQL development skills.
• Strong Python programming and scripting experience for data engineering and automation.
• Working knowledge of Java and/or Scala, especially in Spark or big data processing environments.
• Experience with shell scripting or automation scripting is an asset.
• Strong debugging, problem-solving, and performance tuning skills.
• Hands-on experience with GitHub for source control, branching, pull requests, code reviews, and release management.
• Experience building or working with CI/CD pipelines for data engineering delivery.
• Experience with DevOps practices, automated testing, deployment automation, and environment promotion.
• Familiarity with Terraform, infrastructure-as-code, or cloud deployment automation is an asset.
• Experience working in Agile/Scrum delivery teams.
• Strong communication skills with the ability to explain complex technical concepts to both technical and non-technical stakeholders.
• Ability to translate business requirements into scalable technical solutions.
• Experience leading or actively contributing to technical discussions, design reviews, and implementation planning.
• Collaborative mindset with the ability to work across data, application, platform, DevOps, product, and business teams.
• Demonstrated ownership, accountability, and a track record of successful delivery in enterprise technology environments.

Nice To Haves

• Experience in banking, financial services, regulatory, or highly governed enterprise environments.
• Experience with data lineage, metadata management, data quality frameworks, and observability tools.
• Experience with enterprise reporting, analytics, AI/ML enablement, or operational data products.
• Knowledge of application integration patterns, APIs, microservices, or event-driven architecture.
• University degree in Computer Science, Engineering, Data Engineering, Information Technology, or equivalent practical experience.

Responsibilities

• Design, build, test, deploy, and support scalable data pipelines across Azure and Databricks environments.
• Develop ETL and ELT processes to ingest, transform, validate, and distribute structured, semi-structured, and unstructured data.
• Build and optimise data pipelines using Azure cloud services, Databricks, Spark, Unity Catalog, dbt, and related data engineering tools.
• Work with stakeholders, product managers, architects, and platform teams to understand business requirements and translate them into reliable technical solutions.
• Design ingestion patterns and onboard new data sources into the enterprise cloud data platform.
• Implement data quality, reconciliation, validation, lineage, and observability capabilities to ensure data accuracy, reliability, and traceability.
• Develop reusable data engineering frameworks, patterns, and standards to improve delivery efficiency and operational stability.
• Support data governance and access control through Unity Catalog, platform security standards, and enterprise risk management practices.
• Create and maintain technical design documentation, including logical and physical data flow views, pipeline designs, operational runbooks, and implementation details.
• Drive adoption of DevOps and engineering best practices, including GitHub-based source control, CI/CD pipelines, automated testing, code reviews, and deployment governance.
• Troubleshoot production issues, perform root cause analysis, and continuously improve pipeline performance, reliability, and scalability.
• Collaborate with DevOps, Scrum, product, application, and business teams to deliver data products in an Agile delivery model.
• Contribute to roadmap planning, delivery tracking, technical discussions, and stakeholder communications where required.

Benefits

• Upskilling through online courses, cross-functional development opportunities, and tuition assistance.
• Competitive Rewards program including bonus, flexible vacation, personal, sick days and benefits will start on day one.
• Free tea & coffee, universal washrooms, and lots of space for team collaboration.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed