Senior Cloud Data Platform Engineer

Questrade Financial Group•Toronto, ON

4d•CA$130,000 - CA$150,000•Hybrid

About The Position

Questrade Financial Group (QFG) is seeking a Senior Cloud Data Platform Engineer to join their Enterprise Shared Services (Quest Corp) team. This role is crucial for maintaining and evolving the company's real-time data infrastructure, operating in a hybrid cloud environment spanning on-premises data centers and Google Cloud Platform. The engineer will be responsible for the streaming backbone that powers Questrade's market data, event-driven architectures, and operational analytics. This is a specialized role for individuals with deep expertise in low-latency, high-throughput data systems, particularly Apache Kafka, Change Data Capture (CDC), and OLAP databases like ClickHouse. The position emphasizes ownership, proactive problem-solving, and a commitment to continuous improvement and innovation, aligning with Questrade's mission to help Canadians grow and protect their wealth.

Requirements

Minimum 3+ years of data engineering experience, with at least 2 years focused on streaming and event-driven architectures
Strong expertise in Apache Kafka: cluster operations, topic design, Kafka Connect, Kafka Streams, Schema Registry (Avro/Protobuf), consumer group management
Hands-on experience with Change Data Capture (CDC) patterns and tooling — particularly Debezium, Kafka Connect CDC connectors, Datastream and event sourcing principles
Experience with ClickHouse: MergeTree engine family, materialized views, distributed table design, query optimization, compression codecs, and performance tuning for real-time analytics
Proficiency in SQL (including ClickHouse SQL dialect) and Python
Experience with any of the major cloud platforms (GCP preferred; AWS or Azure also considered)
Strong knowledge of message broker and streaming systems (Kafka, Google Pub/Sub, Kafka MirrorMaker)
Experience with CI/CD pipeline tooling and development practices
Experience building Terraform scripts for infrastructure-as-code
Good knowledge of Airflow operators and DAG configuration
Good knowledge of popular data standards and formats (JSON, Avro, Protobuf, Parquet, ORC)
Data modeling skills with experience in both streaming and batch paradigms

Nice To Haves

GCP data platform knowledge is a strong asset: Dataflow, Dataproc, Cloud Pub/Sub, Cloud Composer, BigQuery, CloudSQL
Experience with stream processing frameworks (Apache Flink, Spark Structured Streaming, or GCP Dataflow) is a strong asset
Experience with OLAP databases beyond ClickHouse (BigQuery, Druid, Redshift) is a plus
Experience with ML Model Training using Google Vertex AI is an asset
Experience in the financial services industry is an asset

Responsibilities

Design, build, and operate high-throughput, low-latency Kafka streaming pipelines for market data, transactional events, and operational telemetry
Architect and implement Change Data Capture (CDC) solutions using Debezium, Kafka Connect, and related tooling to enable real-time data synchronization across operational and analytical systems
Build, tune, and maintain ClickHouse OLAP clusters — including schema design, materialized views, MergeTree engine family selection, distributed query optimization, and data compression strategies
Create and maintain optimal data pipeline architecture across streaming and batch workloads
Design and create new features and automated solutions that meet functional and non-functional business requirements
Identify, design, and implement internal process improvements: automating manual processes, implementing data governance rules, re-designing infrastructure for greater scalability
Maintain in-depth understanding of data protection and network security rules on Cloud Infrastructure
Monitor and manage Kafka cluster health, consumer lag, throughput, schema governance (Schema Registry), and topic lifecycle management
Expand and increase data platform capabilities to resolve new streaming data problems by identifying, sourcing, and integrating new features and technologies
Work with stakeholders to support data-related technical issues and data infrastructure needs
Peer review code and promote DevOps culture within the data teams
Apply ownership in practice: if you see a problem in the platform, do not wait for it to be assigned — awareness is the first act of responsibility
Explore AI integrations into the Data Engineering development cycle
Have a good understanding of data platform best practices to achieve economies of scale, cost reduction, and efficiencies.