Azure Data Engineer

Cynet Systems • Jackson, MI

16h

About The Position

We are seeking an experienced Azure Data Engineer with a strong background in designing and implementing data solutions on the Azure platform. The ideal candidate will have hands-on experience with Azure Synapse Analytics, Azure Data Factory, Databricks, and various other Azure services. This role involves developing robust data pipelines, managing deployments, and ensuring data integrity through efficient processing and troubleshooting.

Requirements

• Hands-on Experience in Azure Synapse Analytics, Azure Data Factory and Data Bricks, Azure Storage, Azure Key Vault, SQL Pools CI/CD Pipeline Designing and other Azure services like functions, logic apps - Linked services, Various Runtimes, Datasets, Pipelines, Activities.
• Strong Hands-on Experience in Various Activities like Control flow logic and conditions (For Each, if, switch, until), Lookup, Stored procedure, scripts, validations, Copy Data, Data flow, Azure functions, Notebooks, SQL Pool Stored procedures etc.
• Strong hands on exp in deployment of code throughout landscape (Dev -> QA -> Prod), Git Hub, CI/CD pipelines etc.
• Strong hands on creating the SQL stored procedures - Functions, Stored Procedures, how to call one SP into another, how to process record-by-record - Dynamic SQL - Data Bricks.
• Strong background about the Python libraries like PySpark, Pandas, NumPy, MySQL, Oracle, PySpark libraries.
• Must have strong hands on to get data through APIs.
• Must be able to install libraries and help users to troubleshoot issues.
• Must have knowledge to get the data through stored procedures via Python.
• Should be able to debug the Python code
• Hands on experience in Spark Pools, Spark.
• Should be able to merge data/delta loads through Notebooks.
• Must have strong background about the Python libraries and PySpark.
• Hands on Exp on Power BI.

Nice To Haves

• Strong verbal and written communication skills.

Responsibilities

• Design and implement CI/CD Pipelines and other Azure services like functions and logic apps.
• Develop control flow logic and conditions using For Each, if, switch, and until activities.
• Manage deployments across the landscape from Dev to QA to Prod using Git Hub and CI/CD pipelines.
• Create and manage SQL stored procedures, functions, and dynamic SQL.
• Process data record-by-record and call stored procedures within other procedures.
• Retrieve data through APIs and stored procedures via Python.
• Install libraries and troubleshoot user issues.
• Debug Python code and work with Spark Pools.
• Merge data and handle delta loads through Notebooks.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed