Senior Software Engineer

Scotiabank • Toronto, ON

13d • Onsite

About The Position

As a Senior Software Engineer on Scotiabank’s Data & AI Technology team, you will lead the design, development and scaling of our enterprise data catalog and data management solutions leveraging the Databricks ecosystem. In this role, you will play a key part in enabling data discovery and governance across the organization by developing robust, scalable and secure data services. This position is ideal for someone who is ambitious, enjoys mentoring others, thrives on tackling intricate technical issues, is motivated by making an impact, and is enthusiastic about shaping the future of the data platform.

Requirements

• Bachelor’s or master’s degree in computer science, Engineering, or a related field
• Solid understanding of common design patterns, lakehouse architecture, and experience working with backend and frontend development frameworks
• Strong programming skills (Python and/or Java)
• Proven experience with DataBricks, Unity Catalog, Delta Lake and Apache spark
• Experience building or integrating enterprise data catalog tools (e.g. Atlan, Alation, Collibra, DataHub, OpenMetadata)
• Experience designing and building reusable data quality, data profiling and data classification frameworks
• Experience implementing data security and access control models
• Solid understanding of data governance, metadata management, and data lineage concepts
• Experience with cloud platforms (Azure, GCP) and infrastructure-as-code tools (Terraform)
• Familiarity with restful APIs, microservice architecture and distributed designs
• Knowledge of streaming technologies (Kafka, structured streaming)
• Creative and persistent approach to problem-solving and strong analytical skills
• Understanding of product development process and data infrastructure requirements
• 7+ years of experience in developing enterprise cloud native applications, with at least 4 years in enterprise data system focused on DataBricks/Azure
• 2+ years of experience leading or mentoring a team of engineers, with a track record of successful project delivery

Responsibilities

• Design and build scalable components of an enterprise data catalog, using automation to streamline data governance processes
• Integrate enterprise data catalog with Databricks and other data platforms
• Design and develop data quality, data profiling and data classification solutions
• Lead technical design discussions and drive architectural decisions for data management solutions
• Advocate for good engineering and development practices within your squad.
• Collaborate, generate ideas, and develop a deep level of expertise in priority projects
• Evaluate trade-offs between correctness, robustness, performance, and customer impact to ensure we build the right solution
• Identify technical risks and proactively propose scalable, long-term solutions
• Mentor and guide junior engineers
• Communicate with stakeholders, work with product managers to understand the product requirements and ensure that the data infrastructure supports the product goals

Benefits

• Upskilling through online courses, cross-functional development opportunities, and tuition assistance.
• Competitive Rewards program including bonus, flexible vacation, personal, sick days and benefits will start on day one.
• Free tea & coffee, universal washrooms, and lots of space for team collaboration.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior