Big Data Engineer

TMX Group • Toronto, ON

66d

About The Position

The TMX group of companies includes leading global exchanges such as the Toronto Stock Exchange, Montreal Exchange, and numerous innovative organizations enhancing capital markets. United as a global team, we’re connecting cross-functionally, traversing industries and geographies, moving opportunity into action, advancing global economic growth, and propelling progress. Through a rich exchange of ideas, meaningful collaboration, and a nimble operating model, we're powering some of the nation's most critical systems, fueling capital formation and innovation, bringing increased opportunity to business visionaries, product ingenuity to consumers, and career exploration to our team. Ready to be part of the action? We are looking for a seasoned and motivated Big Data Engineer to join our growing team of Enterprise Data and Advanced Analytics Platform. The ideal candidate is an expert data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. They must be self-directed and comfortable supporting the data needs of multiple teams, systems and products across a full spectrum of technologies. This role will report into Senior Manager, Data & Analytics.

Requirements

• University degree, College diploma or relevant job experience in Computer Science, Statistics or Mathematics at minimum.
• 5-8 years of overall IT experience and 4+ years in Big Data technologies including Spark, Hive, Presto, etc.
• Strong knowledge of AWS cloud technology on AWS including S3, EC2 and EMR.
• Knowledge working on Snowflake
• Experience in programming languages like python, scala, shell scripting, java
• Good understanding of data warehousing and data modeling techniques.
• Experience in performance tuning for Spark jobs, Hive tables required.
• Experience with development using Agile methodologies.
• Relevant, highly developed professional and technical expertise, with an expert-level understanding in the field and strong knowledge of the business context.
• Excellent interpersonal and communication skills.
• Excellent analytical and problem solving skills.
• Strong analysis / design experience.
• Meticulous attention to detail.
• Must be a self-starter with ability to follow through on projects assigned.
• Collaborate effectively with internal and external stakeholders, managing expectations and expertly handling all inquiries.
• Act as an effective team player and actively mentor junior staff by conducting technical reviews and leading brainstorming sessions.

Nice To Haves

• Good knowledge in Equity Markets business.
• Experience in Terraform, Ansible is a plus along with any Kafka experience
• Experience designing a technology stack for machine learning is a plus.
• Strategic thinker and ability to multitask

Responsibilities

• Design and maintain robust and efficient data pipeline architecture.
• Construct large, complex data sets to fulfill all functional and non-functional business requirements.
• Identify, design, and implement continuous internal process improvements, including automating manual tasks, optimizing data delivery, and re-designing infrastructure for enhanced scalability and performance.
• Manage and administer a wide range of AWS services, such as S3, RDS, EC2, EMR, VPC, Security Groups, CloudFormation, CloudWatch, Managed Airflow, and Simple Notification Services (SNS).
• Possess experience working with both bulk and streaming data, focusing on ingestion, transformation, and secure storage in cloud-based systems.
• Develop the necessary infrastructure for optimal Extract, Transform, Load (ETL) operations from various data sources utilizing SQL and AWS ‘big data’ technologies.
• Adhere to established development best practices, including meticulous code management, branching, and merging within a GIT repository.
• Be available for on-call support and off-business hours support on a rotational basis.

Benefits

• Excitement - Explore emerging technology and innovation, as well as ventures and digital finance that shape the future of global markets! Experience the movement of the market while grounded in the stability of close to 200 years of success.
• Connection - With site hubs in some of the world’s most multicultural cities, we leverage our size and structure to create rich connections and belonging while experiencing powerful global impact through our work.
• Impact - More than a platform, we use our talents to power mission-critical systems that drive global economic advancement, innovation, and growth. As well, our employee-led Team Impact spreads social good via our giving strategy.
• Wellness - From empathetic leadership to a culture of flexibility and balance, we believe wellness at work creates the maximum yield and a stronger “we”. Plus, with a cloud-first and hybrid workstyle, as well as generous time-off and leaves, we support a life well lived!
• Growth - From a growth mindset in our work, to expansion in our business, TMX is home to action-takers energized by the achievement of ambitious growth.
• Ready to enrich your career with impactful work, leaders who truly care, and the flexibility and programs to help you thrive as part of #TeamTMX ?

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level