Lead Data Engineering Developer

Co-operators • Guelph, ON

93d • Hybrid

About The Position

The Lead Data Engineering Developer drives the technical detail design, oversees the development and optimization of data products, ensuring the reliability and performance of ETL processes. This role involves the design and management of complex data pipelines, to take responsibility for the implementation of solutions that comply with architectural requirements, and to collaborate with key partners to meet business objectives. The lead also mentors juniors, sharing expertise to foster their growth.

Requirements

• 5+ years of experience in data engineering with experience in building modern data platforms and data products.
• 3+ years of experience with distributed computing (ex. Spark) and infrastructure as code (ex. Terraform).
• 3+ years of experience designing and implementing scalable, cloud-based data solutions (Azure preferred).
• Deep understanding of data modeling, ETL processes, data warehousing concepts, and best practices in data engineering and analytics.
• Demonstrate excellent communication skills to effectively collaborate with cross-functional teams, articulate technical concepts to non-technical stakeholders, and ensure clarity in project deliverables.

Nice To Haves

• Expert knowledge of cloud security and networking is an asset.
• Vast hands-on experience with the following Azure PaaS is an asset: Azure Data Lake Storage (ADLS), Azure Databricks, Microsoft Fabric, Azure Data Factory (ADF), Azure Synapse, Event Hub, API Management (APIM), Azure Key Vault, Azure SQL, and Purview.
• Have a good understanding of the backup, disaster recovery, and data recovery strategy and execution with the above services.

Responsibilities

• Lead the development of data solutions, ensuring proper data quality and its monitoring, and proper integration aligned with the medallion architecture.
• Create and maintain data models and architectural diagrams.
• Use data engineering tools for data querying (ex: SQL), data handling (ex: pySpark, stored procedures), data storage (ex: ADLS).
• Lead the design of the semantic layer.
• Proactively apply performance optimizations and monitoring of data solutions, such as alerts.
• Lead and enforce proper documentation, to ensure ongoing maintenance and updates.
• Use DevOps platforms (ex: GitHub) and manage infrastructure deployments through code (ex: Terraform).
• Accountable that the outcome of the projects led meet the business objectives.
• Identify and implement engineering best practices and process improvements to enhance the efficiency and effectiveness of our data solutions.
• Ensure governance and security guidelines are followed.

Benefits

• Training and development opportunities to grow your career.
• Flexible work options and paid time off to support your personal and family needs.
• A holistic approach to your well-being, with physical and mental health programs and a supportive workplace culture.
• Paid volunteer days to give back to your community.
• Competitive salary and incentive programs.
• Comprehensive total rewards package including group retirement savings plans, pension and benefits (e.g., health and wellness, dental, disability and life coverage), mental health support and an employee assistance program.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed
• Number of Employees: 501-1,000 employees