Data Engineer
About The Position
The Data Engineer serves as a technical practitioner and individual contributor responsible for designing, building, and maintaining data pipelines, data models, and infrastructure that enable reliable and efficient data delivery across the organization. They work closely with data scientists, analytics teams, business stakeholders, and platform engineers to understand data requirements and deliver high-quality, well-documented data assets. The candidate will also collaborate with IT Security and Compliance teams to ensure data solutions adhere to security standards, data governance policies, and regulatory requirements. This technical role will contribute across the full data engineering lifecycle, including data ingestion, transformation, storage optimization, and quality assurance. The candidate will build and maintain ETL/ELT processes using modern data stack technologies, implement data modeling best practices, and develop reusable frameworks that accelerate data delivery. They will apply software engineering principles to data development, including version control, code review, unit testing, and CI/CD practices to ensure data assets are production-ready and maintainable. This role will help advance the organization's data engineering capabilities by staying current with emerging technologies, tools, and best practices. The candidate will contribute to the evaluation and adoption of cloud-native data platforms, streaming architectures, and data lakehouse patterns. They will also support AI/ML initiatives by building and maintaining data pipelines that feed machine learning workflows, including feature engineering, data preparation for model training, and integration of model outputs into downstream data products and reporting systems.
Requirements
- 2+ years of experience in data engineering and operations, financial institution experience is an asset
- Working knowledge of modern data architectures, including the practical application and construction of Data Lakehouse, Cloud Data Warehouse, Data Mesh, and Data Fabric environments
- Hands-on experience in developing and optimizing complex ETL/ELT pipelines using PySpark, Spark SQL, and T-SQL within cloud-native environments such as Databricks, Snowflake, or Amazon Redshift
- Technical proficiency in operationalizing machine learning workflows, including the development of feature stores, model serving layers, and automated retraining pipelines
- Experience with Agile work planning and CI/CD platforms (e.g., Azure DevOps, GitHub Enterprise) to automate the deployment and validation of data assets
- Expertise in leveraging AI-accelerated software development tools (e.g., GitHub Copilot, Databricks Assistant, Claude Code) to automate code generation, accelerate unit testing, and refactor legacy ETL logic
- Experience applying AI/ML techniques to automate data engineering tasks, such as metadata extraction, schema mapping, and the creation of self-healing data infrastructure
- Proven track record of designing and implementing end-to-end data transformation and reporting solutions, with a deep understanding of logical and physical data modeling
- Experience with data analytics and reporting tools such as Power BI, Micro Strategy, and SAS
- Hands-on experience implementing technical data governance, including the automated enforcement of data quality rules, metadata management, and data retention policies within the pipeline code
- Excellent verbal and written communication skills (e.g. developing business cases and delivering presentations to senior management)
- Strong analytical and problem-solving skills
- Well organized, innovative with a high level of initiative
- Detail oriented, able to manage several complex processes and tasks with a high level of accuracy
- Demonstrated ability to work independently and deal with changing priorities while meeting tight deadlines
- Strong interpersonal skills with the ability to build relationships and work in a team environment
Responsibilities
- Develop and optimize high-performance, end-to-end data transformation and reporting solutions using PySpark, Spark SQL, and T-SQL within cloud-native environments like Databricks
- Architect and implement complex logical and physical data models that support modern patterns such as Data Lakehouse, Data Mesh, and Data Fabric
- Construct robust ETL/ELT pipelines that facilitate seamless data ingestion and transformation from diverse sources into production-ready data assets
- Build and maintain specialized data structures, including feature stores and automated retraining pipelines, to support the operationalization of machine learning models
- Design test strategy, test plan, and test cases
- Own and maintain complete and current assigned technical documentation
- Collaborate with PMO on project estimation and resourcing
- Participate in project risk assessment to identify potential obstacles ahead of time
- Leverage AI-accelerated development tools (e.g., Databricks Assistant, GitHub Copilot, Claude Code) to automate code generation, unit testing, and the refactoring of legacy ETL processes
- Implement AI-powered automation for pipeline monitoring, utilizing machine learning for anomaly detection and the development of self-healing data infrastructure
- Utilize AI tools to accelerate metadata management and data lineage mapping, ensuring that automated pipelines remain compliant with governance standards
- Operationalize data quality and retention policies directly within data pipelines to ensure the integrity and security of the data lakehouse
- Automate the capture of technical metadata and lineage as part of the standard engineering lifecycle to satisfy regulatory and compliance requirements
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Entry Level
