Lead Data Engineer, (Global Security)

RBC•Toronto, ON

1d•Onsite

About The Position

The Pharos Data & Analytics team within RBC Global Security is looking for a Lead Data Engineer to join our mission of turning security data into actionable intelligence. We build and operate the data platform that powers security decision-making across the enterprise — ingesting data from dozens of sources, transforming it at scale, and delivering curated datasets and metrics to security leaders. As our Lead Data Engineer, you will own the technical direction of a squad of data engineers, drive delivery of high-impact data products, and partner with stakeholders to shape our roadmap. You'll work hands-on with code daily while elevating the team through mentorship, design reviews, and engineering best practices.

Requirements

6+ years hands-on data engineering experience and 2+ years leading or technically mentoring a team.
Strong proficiency in Python and PySpark — you write production code daily.
Exceptional skill in Python API development and software engineering (design patterns, testing, packaging, production-grade code).
Deep experience with Databricks (notebooks, jobs, Unity Catalog, Delta Lake).
Solid SQL skills — writing, optimizing, and debugging complex queries.
Experience designing and operating data pipelines at scale (batch and/or streaming).
Familiarity with cloud platforms (Azure preferred; AWS acceptable).
Experience with Terraform for infrastructure-as-code and resource deployment to manage cloud infrastructure.
Experience with Airflow for workflow orchestration and pipeline scheduling.
Track record of following DevOps/Agile best practices (CI/CD, Git, Jira).
Excellent communication skills — able to translate technical concepts for non-technical stakeholders.

Nice To Haves

Experience with Azure Data services, OpenShift, S3 (on-prem or cloud).
Exposure to ML/AI concepts and MLOps workflows.
Experience building observability/monitoring dashboards (Grafana, Databricks dashboards).
Background in cybersecurity data (vulnerability management, SAST/DAST, asset inventory, etc.).

Responsibilities

Lead a squad of data engineers — set technical direction, unblock the team, and drive sprint delivery.
Partner with product owners and security stakeholders to prioritize the backlog based on business value and team capacity.
Own end-to-end delivery of data products from design through production support.
Design and implement scalable batch and streaming pipelines using PySpark, Databricks, and Delta Lake.
Develop and maintain ETL/ELT workflows ingesting from REST APIs, S3, Kafka, and enterprise databases.
Define data models, enforce schema standards, and ensure data quality.
Optimize pipeline performance, reliability, and cost efficiency.
Champion engineering best practices: code review, testing, CI/CD, observability, and documentation.
Drive DataOps maturity — monitoring, alerting, incident response, and automated data quality checks.
Stay current on emerging data technologies and bring pragmatic innovation to the team.
Foster a collaborative, inclusive, and high-trust team culture.