Lead Data Engineer

Joint Commission•Oakbrook Terrace, IL

About The Position

Drives the design, development, and optimization of enterprise data pipelines and platforms. Focused on building robust, scalable, and secure data solutions using modern cloud technologies. Mentor engineers, collaborate with analytics and business teams, and ensure data engineering best practices are followed to support analytics, reporting, and operational needs.

Requirements

Bachelor’s in computer science, Information Systems, Data Engineering, or related fields.
7–10 years of experience in data engineering, pipeline development, and cloud data platforms.
3+ years of experience leading data engineering teams or projects.
Demonstrated success in delivering enterprise-scale data solutions.
Advanced proficiency in Azure Data Factory, Databricks, Fabric, and MSSQL.
Strong experience with data lake, lakehouse, and data warehouse architectures.
Solid understanding of ETL/ELT processes, data modeling, and metadata management.
Proficiency in SQL and Python; familiarity with cloud platforms and big data technologies.

Nice To Haves

Microsoft Certified: Azure Data Engineer Associate
Databricks Certified Data Engineer
Certified in Governance, Risk and Compliance (ISC²) or similar
Master’s degree in computer science, Information Systems, Data Engineering, or related fields.

Responsibilities

Lead the design, development, and maintenance of scalable data pipelines using Azure Data Factory, Databricks, Foundry, and Fabric.
Build and optimize data ingestion, transformation, and integration processes for diverse data sources.
Develop and manage data lakes, lakehouses, and data warehouses, primarily leveraging MSSQL and cloud-native platforms.
Implement ETL/ELT best practices, ensuring high performance, reliability, and data quality.
Automate data workflows and monitor pipeline health, troubleshooting issues as needed.
Mentor and guide a team of data engineers, fostering technical growth and collaboration.
Work closely with analytics, BI, and business teams to translate requirements into technical solutions.
Promote a culture of innovation, continuous improvement, and knowledge sharing within the team.
Evaluate and implement new data engineering tools and technologies to improve efficiency and scalability.
Establish and enforce standards for code quality, testing, and documentation in data engineering projects.
Champion data quality, reliability, and governance within engineering workflows.
Enable self-service analytics by building reusable data models and accessible data sets.
Collaborate with BI teams to ensure data pipelines meet reporting and dashboard requirements.
Translate business requirements into scalable technical solutions.
Ensure data engineering solutions comply with privacy regulations (e.g., GDPR, HIPAA) and internal security standards.
Monitor and enforce data quality, access controls, and compliance metrics.