Sr. Manager, Data Foundations

Chamberlain Group•Oak Brook, IL

3d•Hybrid

About The Position

This role leads the delivery and day-to-day operation of Chamberlain Group's data engineering, analytics engineering, data quality, and Databricks platform capabilities within the Data and Analytics Foundations group. This hands-on leader manages a blended team of full-time Data Engineers and Analytics Engineers alongside external delivery partners, translating the data foundations roadmap into working pipelines, trusted data products, and a well-governed, cost-efficient lakehouse platform. Success in this role looks like reliable data delivered on predictable timelines, measurable improvement in data quality and platform efficiency, engineers who are growing in their craft, and business stakeholders who trust and increasingly self-serve from the data we publish.

Requirements

Bachelor's Degree in computer science or equivalent relevant work experience.
5+ years of experience in data engineering or analytics engineering, including 2+ years of hands-on delivery on Databricks
Demonstrated experience leading technical teams, whether as a people manager or as a technical lead — including setting technical direction, reviewing others' work, and developing engineers
Recent hands-on delivery experience building and operating production data pipelines and data models on the Databricks platform
Experience working alongside external delivery partners, contractors, or offshore teams, with an understanding of what it takes to get quality, predictable delivery from resources you do not directly employ
Demonstrated hands-on use of AI development tools such as Claude, GitHub Copilot, or Databricks Genie Code to improve the speed and quality of data engineering or analytics work, and a track record of raising adoption of those tools across a team
Experience partnering with stakeholders and working with highly technical developers and analysts, with an ability to earn trust
Significant hands-on depth in Databricks, including Delta Lake, Unity Catalog, medallion architecture, PySpark and Spark SQL, and Lakeflow Jobs orchestration — sufficient to review others' code, challenge a proposed design, and personally debug a failing production pipeline
Working knowledge of Databricks platform administration, including Unity Catalog structure and permissions, compute configuration, and consumption cost management
Advanced SQL and demonstrated data modeling skill, including dimensional modeling and design of curated, consumption-ready data products
Working knowledge of the analytics consumption layer — governed semantic and metric definitions, dashboards, and natural language or AI agent access to data — sufficient to set standards for it and review the team's work
Working knowledge of CI/CD and pipeline-as-code practices for data, including Git-based workflows, automated testing, and promotion across environments
Practical understanding of data quality, observability, and data privacy controls, and how to embed them into delivery rather than bolt them on
Ability to operate as a player/coach — setting technical direction and reviewing others' work while remaining credible and capable in the code
Sound judgment about where AI tooling meaningfully accelerates delivery and where it introduces risk, with the discipline to hold AI-assisted work to the same review, testing, and security standards as any other change
Ability to deal with ambiguity and make quality decisions in a dynamic, fast-paced environment
Strong presentation, written and verbal communication skills with an ability to communicate effectively to all levels of the organization
Insistence on high standards with a strong desire to transform and improve and a bias for action

Nice To Haves

Databricks Certified Data Engineer Associate or Professional
Azure platform certification (e.g., Azure Data Engineer Associate)
Experience directly managing the delivery and performance of external partner, systems integrator, or offshore teams, including work allocation, acceptance of deliverables, and holding partners to agreed quality and throughput
Experience leading both data engineering and analytics engineering disciplines, with clear ownership boundaries between ingestion and the curated and semantic layers
Experience establishing reusable, version-controlled transformation frameworks with automated testing and documentation built in
Experience applying agentic AI development tools, such as Claude Code or comparable tools, to data engineering work including legacy code migration, test generation, and large-scale refactoring
Experience defining team-level standards or enablement for responsible AI tool use in an engineering organization
Experience with streaming and near real-time ingestion patterns, including Spark Structured Streaming and Auto Loader
Experience migrating or modernizing legacy ETL onto a lakehouse architecture
Experience with agile/scrum development methodologies
Azure platform expertise, including one or more certifications
Familiarity with Lakeflow Declarative Pipelines (formerly Delta Live Tables)
Familiarity with Declarative Automation Bundles (formerly Databricks Asset Bundles) or comparable deployment tooling
Experience enabling self-service analytics through governed semantic models and curated, well-documented data products
Familiarity with data catalog, master data management, and data quality tooling

Responsibilities

Lead the design, delivery, and operation of scalable batch and streaming pipelines, curated data products, Databricks AI/BI dashboards, and natural-language query experiences over governed data, personally engaging in solution design, code review, and hands-on development where it accelerates the team.
Manage the day-to-day delivery of a blended team of full-time Data Engineers, Analytics Engineers, and external partner resources — setting priorities, sequencing intake, removing blockers, planning capacity, accepting deliverables, and holding partners accountable to agreed quality, throughput, and responsiveness expectations.
Own the Databricks platform operationally, including Unity Catalog administration, workspace and compute configuration, access provisioning and least-privilege enforcement, secrets management, and cost attribution and consumption optimization.
Set the standards for data classification, masking, and access control that engineers implement on published data, in partnership with Data Governance and Information Security, and verify adherence through review.
Own the quality and coherence of the curated and semantic layer the team publishes — Gold-layer models, enterprise metric definitions, certified datasets, and dashboards — holding the Analytics Engineering discipline to agreed standards and arbitrating metric definition disputes that cannot be settled at the working level.
Establish data quality and observability practices so every pipeline ships with quality rules, monitoring, alerting, lineage, and privacy classification by design, and own production reliability for the data estate, including incident response, root cause analysis, and adherence to agreed service levels.
Engage, align, motivate, and inspire the team to maximize their impact, raising the technical bar through mentoring, pairing, and code review.
Drive adoption of and adherence to engineering standards across full-time and partner engineers alike — version control and pull request practices, automated testing, CI/CD, infrastructure and pipeline-as-code, definition of done, and documentation and runbook standards that reduce key-person risk.
Champion the practical adoption of AI-assisted development tools and agentic workflows across full-time and partner engineers to accelerate development, testing, documentation, and migration; set the expectation that AI-generated code is reviewed and tested to the same standard as human-written code, establish guardrails for data privacy and secrets handling, and sponsor reusable AI assets — shared skills, prompt libraries, and agent configurations — that encode engineering standards and are versioned and adopted team-wide.
Engage business stakeholders and partner with data architecture, governance, data science, and BI colleagues to translate priorities into scoped deliverables, align on data source integration and modeling standards, and provide the Director with input into the multi-year data foundations roadmap.
Lead projects that strengthen data foundations directly, reporting delivery status, risks, and dependencies clearly to senior leadership.
Comply with health and safety guidelines and rules; managers should also ensure compliance across their teams.
Protect Chamberlain Group’s reputation by keeping information confidential.
Maintain professional and technical knowledge by attending educational workshops, reading professional publications, establishing personal networks, and participating in professional societies.
Contribute to the team effort by accomplishing related results and participating on projects as needed.
Motivate and lead a high performance team by attracting, developing, engaging and retaining team members.
Drive the performance management and compensation processes by communicating job expectations, monitoring and evaluating performance, providing feedback and facilitating employee development per the company’s policies.
Maintain transparent communication by appropriately communicating organization information to team through department meetings, one-on-one meetings, appropriate email, IM and regular interpersonal communications.
Lead and motivate individuals and teams to create a workplace culture that is consistent with the CG mission, vision and values.