Lead Data Engineer Platform

Comcast•West Chester, PA

4d•Onsite

About The Position

The Data Engineering Platform, Standards & Enablement team at Comcast is seeking a Lead Data Engineer - Platform to serve as the senior-most technical authority for our enterprise data platform. This Engineer 4 will architect and evolve the next-generation data engineering platform that powers Comcast's data ecosystem across Databricks, AWS, Kafka, and distributed database environments. As a platform leader, you will define strategy, establish standards, and guide global teams to deliver a secure, scalable, and cost-efficient platform enabling mission-critical analytics and data workloads. As the Lead Data Engineer - Platform, you will own the vision and roadmap for Comcast's enterprise data platform. You will partner with Cloud Engineering, Cybersecurity, Enterprise Architecture, and Product Engineering to shape foundational capabilities that support thousands of data pipelines and analytics workloads. This role requires deep technical expertise, strong leadership skills, and the ability to influence across a large engineering organization. You will lead platform strategy, enforce governance, optimize performance, and ensure resiliency while mentoring senior engineers and driving engineering excellence. Due to the nature of the role and the team collaborate, this Engineer 4 role is based in our West Chester, PA office. This position is ineligible for visa sponsorship. To be considered for this role, you must be legally authorized to work in the United States and not require sponsorship for employment now or in the future

Requirements

8-10+ years of experience in Data Engineering, Platform Engineering, Distributed Systems, or Software Engineering.
Deep hands-on expertise in Databricks engineering and administration: Cluster policies & cost governance; Workspace architecture; Unity Catalog & data access models; Delta Lake performance tuning; Job orchestration & pipeline optimization
Advanced AWS experience with: S3 lifecycle tiers, encryption, cross-account architectures; MSK provisioning, tuning, scaling, monitoring; EC2, VPC networking, IAM role design; Lambda for serverless automation
Strong background in backend/datastore technologies (Oracle, MongoDB).
Expert-level knowledge of Kafka, Spark, distributed computing, and real-time ingestion frameworks.
Proficiency in Python, SQL, and Shell scripting.
Prior leadership experience managing platform teams and offshore delivery models.
Proven ability to define and enforce platform standards across large organizations.
Excellent communication and stakeholder influence skills.

Nice To Haves

Experience with Databricks Unity Catalog rollout at enterprise scale.
Experience creating reusable Spark frameworks or platform accelerators.
Familiarity with Databricks Serverless, DLT, or Photon.
Knowledge of Kubernetes, container orchestration, and microservices patterns.
AWS or Databricks professional certifications.
Telecom or large enterprise digital commerce experience.

Responsibilities

Enterprise Platform Strategy & Architecture Define and evolve Comcast's enterprise data platform strategy across Databricks, AWS, Kafka, and database ecosystems.
Develop architectural patterns, reference implementations, reusable frameworks, and platform guardrails.
Lead decisions on compute orchestration, storage tiering, data access models, and multi-region resiliency.
Serve as SME for Databricks architecture, including security model, cluster governance, Unity Catalog, Delta Lake strategy, and optimization frameworks.
Technical Leadership & Engineering Excellence Act as the highest-level escalation point for complex platform issues.
Set engineering standards and coding conventions for all data engineering domains.
Mentor senior engineers and architects; promote best practices and thought leadership.
Evaluate emerging technologies to enhance scalability and operational efficiency.
Platform Operations, Governance & Security Own platform governance, including workspace lifecycle, cluster policy enforcement, and compliance.
Drive adoption of monitoring and observability tools (ELK, CloudWatch, Databricks native logging).
Implement FinOps strategies for cost management and forecasting.
Ensure platform resilience, disaster recovery readiness, and high availability.
Cross-Functional Leadership & Delivery Lead globally distributed platform engineering teams (onshore, offshore, vendor partners).
Partner with Security, Enterprise Architecture, and CloudOps to align standards and governance.
Drive quarterly planning, roadmap execution, and platform enhancements.
Communicate risks, KPIs, and modernization opportunities to executive stakeholders.

Benefits

We believe that benefits should connect you to the support you need when it matters most, and should help you care for those who matter most. That's why we provide an array of options, expert guidance and always-on tools that are personalized to meet the needs of your reality—to help support you physically, financially and emotionally through the big milestones and in your everyday life. Please visit the benefits summary on our careers site for more details.

Stand Out From the Crowd

Upload your resume and get instant feedback on how well it matches this job.

Upload and Match Resume