DataOps/Cloud Data Engineer - Senior

ITProFound Inc • Toronto, ON

15d • Onsite

About The Position

We do have a job opening for DataOps/Cloud Data Engineer - Senior for our direct client Province of Ontario. The position details are given below:

Job Details:
Job ID: RQ11099 - DataOps/Cloud Data Engineer - Senior
Client : Health Services Cluster
Location : 5700 Yonge St. Toronto , Ontario
Duration: 1 year + Extension
Type: Onsite

Job Description:

Responsibilities:

Design, develop, test, implement, and troubleshoot: data pipelines using various tools such as Databricks, Azure Data Factory, AWS Gluecomplex data transformation procedures, data models for data warehouses and lakehouses

Requirements

• Programming & scripting: Python, SQL, Linux shell, PowerShell.
• Data manipulation/analysis using pandas and pyspark.
• Working with XLSX, CSV, JSON files, relational databases, cloud storage, structured and unstructured data
• AWS and/or Azure services.
• Cloud data warehouse solutions such as AWS Redshift.
• Data lakehouse solutions such as Databricks (Delta Lake).
• Data processing orchestration/automation
• ETL tools (such as Azure Data Factory, AWS Glue) as well as cloud agnostic tools such as Informatica IDMC.
• Data modeling & architecture: relational and dimensional modeling.
• Data reporting/visualization.
• Full SDLC from requirements gathering, design, implementation, testing to deployment and production support
• Project management (agile/scrum and waterfall)
• Change and Incident management
• Communication, presentation and negotiation skills
• Consulting, problem-solving and decision-making skills
• Experience working in OPS and/or public sector in general
• Experience using AWS Services such as Glue, StepFunctions, Lambda, S3
• Extract/Transform/Load data using tools such as Informatica IDMC

Responsibilities

• Design, develop, test, implement, and troubleshoot: data pipelines using various tools such as Databricks, Azure Data Factory, AWS Gluecomplex data transformation procedures, data models for data warehouses and lakehouses

What This Job Offers

• Career Level: Senior
• Education Level: No Education Listed