Lead Data Engineer

Ryan Specialty•Chicago, IL

About The Position

The Data Engineer will be responsible for building, modernizing, and maintaining data pipelines that feed Ryan Specialty’s Enterprise Data Hub. This role focuses on transforming complex, legacy system data into high‑quality, analytics‑ready datasets that Power BI, advanced analytics, and AI use cases across RSUM and the broader enterprise. This position plays a key role in large‑scale historical data migrations, system decommissioning, and enabling scalable analytics through reliable, well‑governed data pipelines. What will your job entail? Job Responsibilities:

Requirements

Bachelor’s degree required, Master’s degree preferred; Computer science, Information Technology or Engineering is preferred. Any other related discipline or commensurate work experience considered.
10+ years of experience in Data Engineering or related field with a Bachelor’s degree, or 8 years of experience with a Master’s degree is preferred.
Subject Matter Expert in SQL Server Management, Azure Data Factory Development, Databricks Engineering, Delta Parquet File Management, and SQL Server Integration Services.
Proficiency in Profisee MDM Integration, Change Data Capture implementation, and data quality and governance with Azure Purview.
Proven experience in Tableau and BI Reporting, and strong skills in Automation and Process
Big Data Analytics
Leadership
Data Governance
Data Modeling
Data Processing
Data Warehousing
Database Management
Knowledge of ETL processes
Machine Learning
Predictive Analysis
Programming Languages
Critical Thinker
Communication
Relationship Building
Effective Communication
Time Management
Problem Solving
Attention to Detail

Nice To Haves

Advanced Data Analytics Certifications (ADAC), SAS Statistical Business Analyst Professional Certification (SBAPC), or Cloudera Certified Professional (CCP) Data Engineer a plus.

Responsibilities

Design, develop, and maintain data ingestion and transformation pipelines into the Enterprise Data Hub (EDH).
Build custom ETL/ELT processes to migrate historical data from legacy policy administration systems (IMS, C1, etc.).
Implement data quality checks, validations, and monitoring to ensure trusted datasets.
Collaborate with data architecture to align pipelines with canonical data models and governance standards.
Support BI, analytics, and AI use cases by delivering clean, timely, and well‑structured data.
Assist in decommissioning legacy data infrastructure by redirecting pipelines to EDH.
Troubleshoot pipeline failures and performance issues in a production environment.
Document data flows, transformations, and dependencies.
Mentor less experienced data engineers and team members, fostering a collaborative and innovative work environment.
Explore and implement new technologies and practices in data engineering to continually improve our data processes and infrastructure.
Design, build, and maintain scalable and efficient data pipelines, implementing both batch and real-time data integration processes.
Develop and optimize Databricks notebooks for big data processing and analytics.
Implement and manage data storage solutions, optimizing for query performance and cost-efficiency.
Integrate and configure Master Data Management tools, focusing on data governance and quality.
Set up and manage Change Data Capture processes for real-time data synchronization.
Implement data cataloging, classification, and lineage, developing data governance policies, and ensuring compliance.
Enhance and support reporting solutions, developing complex dashboards, and integrating with data services.
Drive automation and process optimization and scripting and implement CI/CD pipelines.

Benefits

Ryan Specialty seeks to offer our employees a comprehensive and best-in-class benefits package that helps them — and their family members — achieve their physical, financial, and emotional well-being goals.
In addition to paid time off for company holidays, vacation, sick and personal days, Ryan offers paid parental leave, mental health services and more.
For additional information on Ryan Specialty Total Rewards, visit our website https://benefits.ryansg.com/.
We provide individuals with disabilities reasonable accommodations to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment in accordance with applicable law.

Stand Out From the Crowd

Upload your resume and get instant feedback on how well it matches this job.

Upload and Match Resume