Data Engineering Manager

Gallagher•Rolling Meadows, IL

62d

About The Position

The Data Engineering Lead is a hands-on technical leader responsible for designing, building, and supporting the data platform that powers analytics, reporting, and AI at Gallagher. The role leads complex data engineering initiatives while remaining deeply involved in development, integrations, production troubleshooting, performance optimization, code reviews, and engineering standards. It is an individual-contributor role with technical leadership responsibilities and no direct people management. The role works closely with data engineering, analytics, data science, business, and technology teams to translate business needs into scalable, reliable data solutions. The position develops and supports ETL/ELT pipelines and data integrations; creates curated datasets and data products; and builds data foundations for analytics, AI, and advanced analytics. It provides technical direction, establishes engineering practices, ensures production reliability, and partners across the organization on business-aligned data solutions, security, governance, and access controls.

Requirements

Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
7+ years of hands-on data engineering experience, including 3+ years in a technical lead or senior engineering role.
Strong hands-on experience with Snowflake, Azure Data Factory, Databricks/Spark, ADLS Gen2, and SQL.
Strong programming experience with Python, Scala, or an equivalent language.
Proven experience designing and implementing enterprise ETL/ELT pipelines and data integrations.
Strong understanding of data lake, data warehouse, and data modeling concepts.
Experience with production troubleshooting, performance tuning, data quality, and pipeline reliability.
Experience with CI/CD and modern software engineering practices.
Ability to lead technical initiatives and influence engineers without direct management authority.
Strong problem-solving and communication skills.

Nice To Haves

Experience modernizing legacy data platforms and integrations.
Experience with data observability, lineage, metadata, or governance.
Experience supporting AI/ML, RAG, semantic search, or other advanced analytics workloads.
Experience optimizing cloud data platforms for performance and cost.
Experience working with distributed engineering teams.

Responsibilities

Design, develop, test, deploy, and support scalable ETL/ELT pipelines and data integrations.
Write production-quality SQL and Python, Scala, or Spark code for data ingestion, transformation, and processing.
Build reusable, modular, and reliable data engineering frameworks and pipelines.
Work across Snowflake, Azure Data Factory, Databricks, Spark, ADLS Gen2, Azure Synapse, and SQL Server/Azure SQL.
Develop curated datasets and data products for analytics, reporting, data science, and AI.
Troubleshoot complex data, pipeline, integration, and performance issues.
Optimize pipelines, queries, compute, and storage for performance, reliability, and cost.
Lead the technical design and implementation of complex data engineering initiatives.
Translate business requirements into practical, scalable technical solutions.
Establish engineering standards for coding, testing, data quality, documentation, security, and deployment.
Conduct design and code reviews and establish reusable engineering patterns.
Mentor engineers and provide technical direction without direct management responsibility.
Identify technical risks, dependencies, and opportunities to simplify or modernize existing solutions.
Design scalable data lake, warehouse, and integration solutions using appropriate data modeling practices.
Build data quality, monitoring, logging, and observability into production pipelines.
Own the technical reliability of data pipelines and integrations.
Lead root-cause analysis and implement permanent fixes for production issues.
Improve CI/CD, release management, automation, and operational support practices.
Help reduce technical debt, manual processes, duplicate integrations, and single points of failure.
Partner with business, analytics, data science, security, and technology teams to deliver solutions aligned with business priorities.
Communicate technical decisions, risks, and tradeoffs clearly to technical and non-technical stakeholders.
Build data foundations for AI and advanced analytics, including structured and unstructured data use cases.
Apply appropriate security, governance, and access controls to enterprise data.