Senior Manager, Data Engineering

Blue Cross Blue Shield Association•Chicago, IL

3d•$131,908 - $178,386•Hybrid

About The Position

The Senior Manager, Data Product Engineering is a hands-on technical leader who leads the design, development, and delivery of data products, pipelines, and analytics solutions that support BCBSA's analytics, reporting, and AI/ML workloads. This role leads a focused engineering team that builds and operates components of the broader data platform — using AWS, Databricks, and Snowflake as the primary stack, alongside modern orchestration, observability, and governance tooling. This is an execution-focused leadership role. This role is expected to write production code, contribute to data architecture and design decisions, conduct code reviews, troubleshoot complex pipeline issues, and lead production support for their team's workloads — while managing and developing a team of data engineers, coordinating with vendor delivery partners, and applying the engineering standards set by leadership and broader architecture team. This role partners with peers across data engineering, analytics, and product teams to deliver assigned data initiatives on time, with quality, and within established platform patterns.

Requirements

BS or equivalent experience
8+ years of experience in ETL, data engineering, data warehousing, or large-scale data platform development.
Minimum 3 years of hands-on experience with AWS, Databricks and Snowflake
Experience managing offshore, nearshore, vendor, or managed services delivery models.
Demonstrated experience in a hands-on data engineering role with active participation in solution design, coding, code reviews, testing, deployment, and production support.
Strong hands-on development experience with SQL, Python, PySpark, Spark, Databricks notebooks/jobs, Snowflake SQL, and AWS data services.
Proven experience designing and operating ETL/ELT pipelines in enterprise environments.
Experience leading data engineering teams and mentoring engineers on technical delivery and best practices.
Experience with CI/CD, DevOps, Git-based development, automated testing, monitoring, and deployment practices.
Experience working in Agile, Scrum, SAFe, or product-oriented delivery environments.
Experience with data observability, platform monitoring, FinOps, and cost optimization practices.
Certified Data Engineer Associate - Databricks or Professional level
SAFe Agilist Certification (SA) - Scaled Agile, Inc
AWS Certified Solution Architect - Amazon Web Services (AWS) or AWS Certified Cloud Practitioner
Strong understanding of cloud-native data architecture, data lakes, lakehouse architecture, data warehouses, data marts, and dimensional modeling.
Strong knowledge of data governance, data quality, metadata management, lineage, access control, and production support processes.
Working understanding of SOC 2, HIPAA, and HITRUST; experience building and delivering data engineering pipelines under regulated data handling.
Strong partnership skills across Engineering, Product, Analytics, Security, and external technical alliances.
Familiarity with HIPAA, PHI, PII, data privacy, security, and regulatory compliance requirements.
Prior hands-on data product engineering experience with NDW, CCL, and VBP data — including ingestion, transformation, curation, and downstream data product development

Nice To Haves

MS degree

Responsibilities

Lead a team of data engineers in the design, development, and delivery of scalable data pipelines and data products using AWS, Databricks, Snowflake: Spark, PySpark, Python, and SQL — contributing as a hands-on engineer alongside the team.
Engage across the complete software development lifecycle for assigned initiatives — requirements analysis, estimation, technical design, development, code review, testing, release planning, deployment, and post-production support.
Contribute to architecture and design decisions within the scope of owned data products; apply enterprise technical standards, reference architectures, and platform patterns set by Architecture and Leadership.
Identify opportunities for product modernization, reusability, and engineering improvements, and bring forward recommendations.
Lead end-to-end engineering delivery for NDW/VBP/CCL data product functions.
Lead and hands-on contribute to the development of reliable, scalable, and high-performance ETL/ELT pipelines supporting batch, near-real-time, and analytical workloads for assigned data products.
Build and operate ingestion, transformation, and curation patterns aligned to medallion architecture, lakehouse, and dimensional modeling principles established by the broader data platform.
Review production code from team members and vendor partners, apply established coding standards, promote reuse of common frameworks, and ensure maintainability, scalability, and reliability of delivered solutions.
Troubleshoot and resolve pipeline failures, data quality issues, and performance bottlenecks; partner with platform, infrastructure, and cloud engineering teams on complex incidents that span beyond the team's scope.
Build solutions on AWS-native services and leverage Databricks and Snowflake as core components for analytics and ML workloads, following established platform architecture patterns.
Implement and maintain the CI/CD lifecycle for the team's data pipelines: Git-based development, automated testing, deployment automation, infrastructure as code, and rollback patterns — aligned with enterprise DevOps standards.
Optimize compute, storage, and workload execution across AWS, Databricks, and Snowflake for assigned workloads; apply FinOps practices in day-to-day engineering and surface cost optimization opportunities.
Implement monitoring, alerting, observability, performance tuning, and production readiness practices for the team's data products in line with platform-wide SLAs and standards.
Deliver data product engineering work that powers BCBSA data products across claims, member, provider, pharmacy, clinical, financial, operational, regulatory, and value-based care domains.
Bring deep, hands-on expertise across NDW, CCL, and adjacent BCBSA enterprise data assets — applying that knowledge to data model design, source-to-target mapping, lineage, and downstream data product development.
Partner with product managers, analytics, and data science teams to build curated datasets, semantic models, and reusable data products that support Medicare Advantage, Risk Adjustment, Stars/HEDIS, Cost of Care, and member experience use cases.
Treat data as a product — applying product thinking to schema design, data contracts, consumer experience, documentation, versioning, and lifecycle management.
Build data quality, lineage, and metadata capture into pipelines and data products as standard engineering practice; address data quality issues at the source rather than downstream.
Apply HIPAA, PHI/PII protection, access control, and regulatory requirements in day-to-day engineering; partner with Privacy, Security, Compliance, and Data Governance teams on controls, reviews, and remediation for data products handling sensitive information.
Manage day-to-day vendor relationships, delivery commitments, and performance for the team's third-party engineers and managed services partners; escalate issues and risks as appropriate.
Coordinate offshore, nearshore, and hybrid delivery teams — driving quality, velocity, and accountability through clear assignments, code reviews, and delivery checkpoints.
Provide input to sourcing, finance, and architecture on contract scoping, SOW review, and vendor performance — under the direction of leadership.
Manage, mentor, and develop a team of data engineers — including performance management, coaching, technical guidance, day-to-day prioritization, and career development.
Foster a strong engineering culture on the team grounded in code quality, operational excellence, ownership, and continuous learning.
Contribute to engineering practices, mentoring, and knowledge sharing across the broader data engineering organization.