Sr Data Architect, Databricks

McKesson•Columbus, OH

1d•$124,100 - $206,900

About The Position

McKesson is an impact-driven, Fortune 10 company that touches virtually every aspect of healthcare. We are known for delivering insights, products, and services that make quality care more accessible and affordable. Here, we focus on the health, happiness, and well-being of you and those we serve – we care. What you do at McKesson matters. We foster a culture where you can grow, make an impact, and are empowered to bring new ideas. Together, we thrive as we shape the future of health for patients, our communities, and our people. If you want to be part of tomorrow’s health today, we want to hear from you. CoverMyMeds' Data & Analytics organization is looking for a Lead Data Architect to define and advance our enterprise data engineering strategy on the Databricks lakehouse platform. Our Data Engineering team is a highly technical group of results-driven engineers and architects focused on delivering high-quality, repeatable, and scalable data solutions. Together with our business units, this work ultimately helps more people get the medicine they need to live healthier lives. McKesson complies with all applicable U.S. immigration laws and regulations. The Company does not provide employer support or sponsorship for any immigration related employment benefit for this role. Applicants must be currently authorized to work in the United States on a fulltime basis without the need for employer support or sponsorship now or in the future. This includes having the legal right to work in the United States without the need for McKesson support or sponsorship for any immigration related employment authorization (e.g., H1B, O1, E3, H1B1, TN, F1 OPT, F1 STEM OPT, F1 CPT, etc.) now or in the future. If you will require McKesson to provide immigration support or sponsorship now or in the future, you should not apply for this position. What You'll Do: The Lead Data Architect – Databricks holds a critical, senior-level role directly influencing CoverMyMeds' modern data platform strategy. This position owns the architecture of our Databricks-on-Azure environment end to end: from foundational platform design through Unity Catalog governance, CI/CD deployment patterns, security architecture, and medallion lakehouse standards—at enterprise scale. The ideal candidate has built and scaled a Databricks platform from the ground up, established enterprise-grade standards across engineering teams, and driven adoption across a complex, regulated organization. This is an architect who remains deeply hands-on while providing technical leadership across multiple product domains and delivery teams.

Requirements

Bachelor's degree in Computer Science, Information Systems, or a related field; advanced degree preferred.
10+ years of experience in data engineering and data architecture, with at least 4 years in a principal or lead architecture capacity.
Demonstrated experience implementing Databricks from the ground up: workspace provisioning, Unity Catalog configuration, metastore design, SCIM integration, and cluster policy governance at enterprise scale.
Deep expertise in Delta Lake: ACID guarantees, Delta log mechanics, schema evolution, time travel, Change Data Feed, and performance optimization techniques.
Strong Azure cloud platform proficiency: ADLS Gen2, Azure Data Factory, Azure Key Vault, Azure Private Endpoints, VNet/NSG design, and Entra ID integration.
Experience designing and operating medallion lakehouse architectures across regulated environments with PHI/PII data.
Proven ability to define and enforce CI/CD deployment standards for data platforms, including branching strategies, environment promotion, approval gates, and audit-trail evidence.
Strong understanding of data governance frameworks: cataloging, lineage, data quality, data rights enforcement, and SailPoint/Okta access provisioning integration.
Excellent communication and stakeholder management skills; ability to translate complex technical architecture into leadership-level decision documents.

Nice To Haves

Healthcare industry experience, including working with PHI, HIPAA compliance, or payer/provider/pharmacy data domains.
Experience building SOX-compliant data platform controls: ITGC change management, logical access evidence, segregation of duties enforcement via CI/CD.
Familiarity with Databricks Asset Bundles, LakeFlow, and Delta Live Tables for end-to-end pipeline delivery.
Experience architecting cross-workspace Unity Catalog federated models and data product sharing via Delta Sharing.
Knowledge of streaming ingestion patterns: Kafka, Databricks Auto Loader, Structured Streaming, and event-driven pipeline designs.
Experience with IaC tooling (Terraform, Azure Bicep) for infrastructure provisioning alongside Databricks platform management.
Familiarity with AI/ML data readiness: feature store patterns, training data pipeline governance, and model data lineage.

Responsibilities

Own end-to-end architecture for the Databricks-on-Azure lakehouse platform, including Unity Catalog design, workspace federation, storage account topology, cluster governance, and compute cost optimization.
Define and enforce medallion architecture standards (Landing → Bronze → Silver → Gold) across all data engineering teams and product domains.
Establish CI/CD deployment patterns for Databricks Asset Bundles, working with DevOps and platform engineering teams to ensure audit-ready, environment-promoted pipelines.
Lead platform scaling decisions: networking (VNet, private endpoints, subnet allocation), workspace growth planning, and multi-subscription design across CMMA and CMMB environments.
Design and govern fine-grained access control patterns using Unity Catalog ABAC, RBAC, and row/column-level security—translating regulatory requirements into reusable platform controls.
Drive performance engineering standards for Delta Lake pipelines, including partitioning strategies, Z-ORDER, liquid clustering, OPTIMIZE/VACUUM cadences, and Photon adoption.
Define ingestion accelerators and reusable pipeline templates for common patterns (CDC via Qlik, ADF, SFTP, Kafka, API) to reduce per-dataset engineering effort.
Partner with SRE, DBA, Data Governance, Legal, and Analytics teams to align platform architecture with organizational compliance, data rights, and consumption requirements.
Provide architectural guidance on PHI/PII data handling, including masking strategies, HIPAA-aligned access controls, and multi-environment data lifecycle policies.
Evaluate and make build-vs-buy decisions across the platform toolchain: ingestion, orchestration, data quality, cataloging, and monitoring.
Mentor senior engineers and data architects, establishing design review processes, ADR practices, and platform documentation standards.