Big Data Lead

HEXAWARE • United States,

20h

About The Position

This role requires expertise on Azure platforms, including Azure Databricks, ADLS, and their various components. The ideal candidate will have in-depth knowledge and hands-on experience in the Microsoft Azure data stack, such as Azure Data Factory, Azure SQL Database, Azure Functions, Logic apps, Spark, and Python programming. A strong understanding of data architecture and data management principles is essential. The position involves designing ETL Pipelines for Batch and Real-time processing using Azure Data Factory, with hands-on experience in Function apps, Event Hubs, Web apps, and APIs. Experience with migrating to Salesforce is a plus. The role demands strong experience in designing ETL/ELT Pipelines/Data flows, primarily using Azure Data Factory, with SSIS/Informatica as additional skills. Proficiency in programming languages like Python, SQL, Spark notebooks, and Scala is required. Strong development and design skills in leading RDBMS environments such as MS SQL Server and Oracle 10g are necessary. Experience with Big data approaches, architectural concepts, data sourcing/ingestion/curation, and storage mechanisms is crucial. Familiarity with real-time/near-real-time Data Warehousing concepts and design approaches is also important.

Requirements

• Expertise on Azure platforms including Azure Databricks, ADLS, and its various components.
• In-depth knowledge and hands-on experience in Microsoft Azure data stack (Azure Data Factory, Azure SQL Database, Azure Functions, Logic apps, Spark, Python).
• Expertise on Azure platforms including Azure Databricks and its various components, including Delta Lake.
• Strong understanding of data architecture and data management principles.
• In-depth knowledge and hands-on experience in Microsoft Azure data stack (Azure Data Factory, Azure Synapse, Azure SQL Database, Azure Functions, Logic apps, Spark).
• In-depth knowledge on designing of ETL Pipelines for Batch and Real time in Azure Data Factory.
• Hands-on experience on Function apps, Event Hubs, Web apps, API M.
• Strong experience with designing ETL/ELT Pipelines/Data flows using Azure Data Factory as Primary skill.
• Proficiency in programming languages such as Python, SQL, Spark notebooks and Scala.
• Strong development & design skills in leading RDBMS environments (MS SQL Server and Oracle 10g).
• Experience of Big data approach, architectural concepts, data sourcing/ingestion/curation and storage mechanisms.
• Experience with real-time / near-real-time Data warehousing concepts and Design approaches.

Nice To Haves

• SSIS/Informatica as additional skills.
• Good to have experience on Migration to Salesforce.

Responsibilities

• Design and develop ETL/ELT Pipelines/Data flows using Azure Data Factory.
• Implement Batch and Real-time ETL Pipelines in Azure Data Factory.
• Utilize Azure Databricks and its components, including Delta Lake.
• Develop solutions using Azure Data Factory, Azure Synapse, Azure SQL Database, Azure Functions, Logic apps, Spark, and Python.
• Work with Function apps, Event Hubs, Web apps, and APIs.
• Lead RDBMS environments, including MS SQL Server and Oracle 10g.
• Apply Big data approaches, architectural concepts, and data management principles.
• Manage data sourcing, ingestion, curation, and storage mechanisms.
• Implement real-time/near-real-time Data Warehousing concepts and design approaches.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed