Principal Engineer, Data Platforms (Architecture & Ingestion)

CVS Health•Work At Home-New Jersey, NJ

2d•$144,200 - $288,400

About The Position

We are seeking a Principal Engineer, Data Platforms to serve as the senior technical authority responsible for the architecture, engineering, governance, and evolution of the Unified Ingestion Platform and Federated Data Lake supporting enterprise observability, security, and operational telemetry data. The Principal Engineer, Data Platforms will define and execute the technology strategy for a highly scalable, cloud-native data platform capable of ingesting and processing hundreds of terabytes of data daily across Azure, AWS, and GCP. The Principal Engineer, Data Platforms will own the enterprise Databricks platform strategy and architecture, including multi-cloud account structures, workspace design, infrastructure automation, security controls, governance frameworks, data lifecycle management, metadata services, and platform standardization. The Principal Engineer, Data Platforms will lead a multi-year transformation initiative focused on consolidating fragmented observability, security, and operational telemetry platforms into a unified, governed, and cost-efficient ecosystem leveraging Databricks, Delta Lake, Unity Catalog, and modern cloud-native engineering practices. As the organization's foremost technical expert in enterprise data platforms, the Principal Engineer, Data Platforms will establish engineering standards, drive infrastructure-as-code and policy-as-code adoption, lead enterprise-wide architecture decisions, and influence strategic technology investments. The Principal Engineer, Data Platforms will partner with engineering, security, operations, infrastructure, and business stakeholders to deliver a resilient, scalable, secure, and future-ready data platform that improves operational intelligence, strengthens governance, accelerates onboarding, enables AI/ML and analytics workloads, and promotes consistent multi-cloud platform operations across the enterprise.

Requirements

10+ years of experience architecting, building, and operating enterprise-scale cloud data platforms, data lakes, distributed systems, and large-scale data ingestion ecosystems supporting mission-critical workloads.
8+ years of experience designing and administering Databricks platforms, including Apache Spark, Delta Lake, Unity Catalog, secure data access patterns, metadata management, lineage, and enterprise governance frameworks.
8+ years of experience implementing Infrastructure-as-Code, platform automation, CI/CD pipelines, and Policy-as-Code controls using technologies such as Terraform, GitHub Actions, or comparable cloud-native automation platforms.
7+ years of experience designing and operating high-scale streaming and telemetry ingestion architectures processing more than 100 TB per day using technologies such as Kafka, Event Hub, Kinesis, Pub/Sub, NiFi, Cribl, or similar platforms.
5+ years of experience serving as a Principal Engineer, Lead Architect, or equivalent technical leadership role driving enterprise platform strategy, multi-cloud architecture across Azure, AWS, and GCP, engineering standards, security governance, and large-scale transformation initiatives resulting in measurable improvements in scalability, reliability, operational efficiency, governance maturity, and platform adoption.
Bachelor’s degree from accredited university or equivalent work experience (HS diploma + 4 years relevant experience).

Nice To Haves

Extensive experience building and evolving enterprise-scale data platforms, analytics ecosystems, observability platforms, or cloud-native engineering platforms within large and highly regulated environments.
Hands-on expertise with Unity Catalog, Delta Sharing, metadata services, lineage frameworks, information stewardship, retention controls, and federated operating models.
Proven success driving Splunk rationalization efforts, security analytics modernization, SIEM ecosystems, telemetry consolidation, or monitoring transformation programs.
Strong background designing secure serverless computing environments, multi-tenant architectures, workload isolation strategies, and Databricks deployments spanning Azure, AWS, and GCP.
Demonstrated success establishing FinOps disciplines, consumption governance, chargeback methodologies, provisioning automation, and resource optimization programs that improve efficiency and operational simplicity.
Databricks certifications (Data Engineer, Data Architect, Platform Architect, or equivalent) coupled with knowledge of AI/ML enablement, advanced analytical workloads, OpenTelemetry (OTEL), and Open Cybersecurity Schema Framework (OCSF) standards.

Responsibilities

Define and own the architecture, strategy, and multi-year roadmap for the Unified Ingestion Platform and Federated Data Lake supporting observability, security, and operational telemetry data across the enterprise.
Architect and scale high-volume ingestion and processing capabilities supporting 250+ TB per day across logs, metrics, traces, security events, and operational datasets while driving platform modernization and convergence initiatives.
Establish and govern enterprise Databricks architecture, including multi-cloud account structures, workspace strategies, environment isolation, metadata management, data governance, lineage, and secure access models.
Lead implementation of Infrastructure-as-Code, Policy-as-Code, automated provisioning, and enterprise-grade CI/CD pipelines to enable reliable, standardized, and scalable platform operations.
Define enterprise security, governance, observability, reliability, and operational excellence practices while aligning platform standards with OpenTelemetry (OTEL), Open Cybersecurity Schema Framework (OCSF), and enterprise compliance requirements.
Serve as the highest-level technical escalation point and provide architectural leadership across engineering, security, infrastructure, and platform teams.
Establish FinOps strategies, cost allocation frameworks, resource optimization practices, and capacity planning models to ensure efficient and cost-effective platform operations.
Mentor engineers and architects while driving adoption of cloud-native engineering principles, automation frameworks, governance standards, and platform best practices.
Evaluate emerging technologies and develop strategic roadmaps that improve platform scalability, reliability, governance, developer productivity, and operational efficiency.
Enable advanced analytics, security analytics, observability intelligence, and AI/ML workloads through trusted, governed, and reusable enterprise data platform capabilities.