Principal Data Engineer

Kemper•Downers Grove, IL

3d•Hybrid

About The Position

The Principal Data Engineer serves as a senior technical leader responsible for architecting, developing, and governing enterprise-scale data platforms, pipelines, and analytics solutions. This role provides hands-on technical leadership across data engineering initiatives, cloud modernization efforts, real-time integrations, and enterprise data strategy. The ideal candidate combines deep expertise in modern data engineering technologies with strong leadership capabilities to guide engineering teams, establish best practices, and deliver scalable, secure, and high-performing data solutions that support analytics, reporting, AI/ML, and operational business functions.

Requirements

10+ years of experience in data engineering, data architecture, or software engineering.
Expert-level experience with SQL, Python, Snowflake, and enterprise ETL/ELT frameworks.
Hands-on experience with cloud-native data engineering tools and platforms (e.g., AWS Glue, S3, Snowflake, Kafka, Airflow).
Proven experience leading large-scale enterprise data initiatives and mentoring engineering teams.
Strong understanding of data governance, security, scalability, and performance optimization.
Experience working in regulated industries and understanding data privacy, security, and compliance frameworks.
Strong understanding of insurance industry data (especially P&C and Life domains), including data from policy admin systems (e.g., Guidewire, Life/400), claims platforms, and actuarial models.
Bachelor’s or master’s degree in computer science, Engineering, Information Systems, Data Science, or related fields or equivalent work experience.

Nice To Haves

Insurance industry experience (P&C and/or Life) preferred.
Experience with real-time streaming and event-driven architecture a plus.
Experience in Spark, Kafka, Airflow, DBT, and Infrastructure as Code frameworks preferred.
Familiarity with DevOps and CI/CD pipelines for data engineering platforms.
Experience in working with IDMC/IICS or DBT a plus
Experience in Data warehousing with Data vault 2.0 preferred
Knowledge of Git for version control and collaboration.

Responsibilities

Enterprise Data Architecture & Engineering: Design, development, and optimize enterprise-scale data pipelines and integration frameworks supporting analytics, reporting, operational, and AI/ML workloads. Architect scalable data lake, warehouse, and real-time streaming solutions using cloud-native technologies. Design and maintain logical and physical data models aligned with enterprise architecture standards and normalization best practices. Build robust ingestion, transformation, orchestration, and delivery pipelines across structured and semi-structured data sources. Architect cross-functional data solutions that integrate data from core insurance systems (e.g., policy admin, claims, billing, CRM) and third-party sources.
Technical Leadership: Serve as the technical lead for data engineering initiatives and provide architectural guidance across engineering teams. Mentor and coach junior and mid-level engineers while promoting engineering excellence and continuous improvement. Establish best practices for coding standards, CI/CD, infrastructure as code, monitoring, observability, and operational support. Lead design reviews, technical solutioning sessions, and enterprise architecture discussions.
Cloud & Platform Engineering: Develop cloud-native data solutions using AWS, Azure, and modern data platforms including Snowflake, Spark, Kafka, Airflow, Glue, and related technologies. Drive modernization initiatives involving hybrid-cloud and multi-cloud architectures. Build reusable frameworks and automation solutions to improve scalability, reliability, and engineering productivity.
Data Integration & Processing: Integrate enterprise data from core operational systems, third-party vendors, APIs, and streaming platforms. Develop and optimize ETL/ELT pipelines using SQL, Informatica/IICS, Python, Spark, and cloud-native processing tools. Ensure high-performance query optimization, workload tuning, and efficient data processing across enterprise platforms.
Governance, Security & Compliance: Ensure compliance with enterprise security standards, governance policies, and regulatory requirements including HIPAA, SOX, GDPR, and NAIC standards applicable. Implement data quality, metadata management, lineage, auditing, and observability capabilities. Partner with cybersecurity, governance, and compliance teams to enforce secure and compliant data engineering practices.
Leadership & Collaboration: Collaborate with architects, analysts, actuaries, data scientists, developers, and business stakeholders to deliver scalable and trusted data solutions. Translate complex business requirements into enterprise data architectures and engineering solutions. Support strategic initiatives including underwriting analytics, claims automation, customer analytics, and regulatory reporting. Provide technical mentorship and architectural oversight to junior and mid-level engineers across teams.