Lead Azure Data Engineer

Ryan Specialty•Carmel, IN

22h•$140,000 - $175,000

About The Position

The Lead Azure Data Engineer reports to the VP of Data Engineering and is responsible for implementing data solutions in collaboration with Business Analysts, Data Analysts, IT teams, and key stakeholders. The role leads projects and coaches junior staff in building and optimizing data pipelines, integrating data from various sources, managing ETL processes, and ensuring data quality and consistency. It promotes data engineering practices aligned with current technologies and industry best practices, with a focus on performance, reliability, and efficiency. The position provides technical leadership across data architecture and engineering and works with stakeholders to align solutions with business objectives. It manages and enhances SSIS packages and SSRS reports; builds scalable batch and real-time pipelines with Azure Data Factory; develops Databricks notebooks; manages Delta Parquet storage in Azure Data Lake Storage; and integrates Profisee MDM tools. It also supports change data capture, governance and lineage using Azure Purview, Tableau reporting, automation and CI/CD with Azure DevOps, and SQL Server optimization. A bachelor's degree is required, with a master's degree preferred; relevant disciplines include computer science, information technology, and engineering. Related disciplines or commensurate work experience may be considered. Preferred experience is 12+ years in data engineering with a bachelor's degree or 8 years with a master's degree. The role requires expertise across SQL Server, Azure data engineering tools, ETL, data governance, and BI reporting. Advanced data analytics, SAS Statistical Business Analyst Professional, or Cloudera Certified Professional Data Engineer certifications are a plus. Applicants must be authorized to work for any U.S. employer; visa sponsorship is not available. Ryan Specialty states that it is an equal opportunity employer committed to a diverse, inclusive, and equitable workplace. The role has a target salary range of $140,000–$175,000 annually. Full-time roles are eligible for bonuses and benefits. The company describes benefits including paid time off for holidays, vacation, sick and personal days, paid parental leave, and mental health services, among other benefits. Reasonable accommodations are available for applicants and employees with disabilities.

Requirements

Bachelor’s degree required; a related discipline or commensurate work experience may be considered.
Preferred experience: 12+ years in Data Engineering with a bachelor’s degree, or 8 years with a master’s degree.
Subject matter expertise in SQL Server Management, Azure Data Factory development, Databricks engineering, Delta Parquet file management, and SQL Server Integration Services.
Proficiency in Profisee MDM integration, Change Data Capture implementation, and data quality and governance with Azure Purview.
Proven experience in Tableau and BI reporting, and strong skills in automation and process optimization.
Skills include big data analytics, leadership, data governance, data modeling, data processing, data warehousing, database management, ETL, machine learning, predictive analysis, and programming languages.
Behavioral skills include critical thinking, communication, relationship building, time management, problem solving, and attention to detail.
Must be authorized to work for any employer in the U.S.; the employer is unable to sponsor or take over sponsorship of an employment visa.

Nice To Haves

Master’s degree preferred; computer science, information technology, or engineering is preferred.
Advanced Data Analytics Certifications (ADAC), SAS Statistical Business Analyst Professional Certification (SBAPC), or Cloudera Certified Professional (CCP) Data Engineer certification.

Responsibilities

Provide technical leadership in data architecture and engineering, collaborating with stakeholders to align data solutions with business objectives.
Mentor less experienced data engineers and team members, fostering a collaborative and innovative work environment.
Explore and implement new data engineering technologies and practices to improve processes and infrastructure.
Manage and enhance SSIS packages for data integration and ETL processes, as well as SSRS reports for business intelligence.
Design, build, and maintain scalable data pipelines using Azure Data Factory for batch and real-time integration.
Develop and optimize Databricks notebooks for big data processing and analytics in Azure Databricks.
Implement and manage Delta Parquet storage in Azure Data Lake Storage, optimizing query performance and cost efficiency.
Integrate and configure Profisee Master Data Management tools within Azure, with a focus on data governance and quality.
Set up and manage Change Data Capture processes for real-time synchronization in Azure data platforms.
Implement data cataloging, classification, and lineage using Azure Purview; develop governance policies and ensure compliance.
Enhance and support Tableau reporting solutions, develop complex dashboards, and integrate them with Azure data services.
Drive automation and process optimization using Azure services and scripting, and implement CI/CD pipelines using Azure DevOps.
Lead SQL Server database optimization and performance tuning efforts.
Lead, coach, and mentor junior staff on projects.