Data Engineer - Databricks, ADF, SQL - Technical Lead

Innosystech•Dallas, TX

11d•Hybrid

About The Position

We are seeking an experienced Data Engineer with a strong technical leadership background to join our team. This role will focus on designing, developing, and implementing enterprise data engineering solutions using Azure Databricks, Azure Data Factory (ADF), SQL, and PySpark. The ideal candidate will have extensive experience in data warehousing, ETL/ELT processes, and leading data engineering teams. This is a long-term contract position based in Dallas, TX, with an onsite/hybrid work model.

Requirements

10+ years of overall experience in Data Engineering, Data Warehousing, ETL, or related data technologies.
5+ years of hands-on experience with Azure Data Factory (ADF) and Azure-based data engineering solutions.
4+ years of strong hands-on experience with Azure Databricks.
Advanced expertise in SQL, including complex queries, stored procedures, performance tuning, and data transformations.
Strong experience designing and developing ETL/ELT pipelines for large-scale enterprise data platforms.
Hands-on experience with PySpark / Spark for distributed data processing.
Experience leading data engineering teams and providing technical direction, code reviews, troubleshooting, and production support.
Strong understanding of data warehousing, dimensional modeling, data integration, and data quality.
Ability to work from Dallas, TX as required.

Nice To Haves

Experience with Azure Data Lake Storage (ADLS Gen2)
Experience with Delta Lake
Experience with Databricks Lakehouse
Experience with Unity Catalog
Experience with CI/CD/Azure DevOps
Experience with Python
Experience with Git
Experience with Power BI

Responsibilities

Lead the design, development, implementation, and support of enterprise data engineering solutions using Azure Databricks, Azure Data Factory, SQL, and PySpark.
Design and optimize scalable ETL/ELT pipelines to ingest, transform, and integrate data from multiple enterprise sources.
Develop Databricks notebooks and Spark-based data processing solutions for high-volume datasets.
Build and maintain ADF pipelines, datasets, linked services, triggers, and integration workflows.
Develop and optimize complex SQL queries and data transformation processes.
Provide technical leadership to the data engineering team, including solution guidance, development standards, code reviews, and mentoring.
Troubleshoot data pipeline failures, performance bottlenecks, and production issues.
Work closely with architects, business analysts, application teams, and data consumers to translate requirements into technical solutions.
Implement data quality, validation, monitoring, logging, and error-handling mechanisms.
Support deployment and release activities across development, QA, and production environments.
Ensure solutions follow enterprise security, governance, performance, and data management standards.