Principal Data Platform Engineer (Databricks)

Prominence Advisors

20h•Hybrid

About The Position

Prominence is looking for a Principal Data Platform Engineer to serve as a senior technical leader responsible for owning both the architecture and delivery of modern data platforms for our clients. This is a highly visible, client-facing role where you will shape data strategy, design scalable solutions, and lead implementation efforts end-to-end. You will partner directly with client stakeholders while mentoring internal team members, ensuring high-quality delivery aligned with Prominence’s standards of excellence. This role is ideal for someone who thrives at the intersection of strategy and execution—equally comfortable whiteboarding architecture with executives as they are building and optimizing pipelines alongside engineering teams.

Requirements

7+ years of experience in data engineering, with demonstrated progression into technical leadership and architecture ownership
Deep expertise in Databricks and modern lakehouse architectures, including Delta Lake and Spark-based processing
Advanced proficiency in SQL and Python, with strong experience building and optimizing large-scale data pipelines
Hands-on experience with cloud platforms such as AWS, Azure, or GCP, including data services and infrastructure design
Strong understanding of data modeling concepts, ETL/ELT patterns, and distributed data processing
Experience with orchestration tools (e.g., Airflow) and transformation frameworks (e.g., dbt)
Proven ability to lead technical delivery while remaining hands-on when needed
Strong client-facing experience, including requirements gathering, solution design, and executive communication
Ability to navigate ambiguity, prioritize effectively, and drive clarity in complex environments
Must be legally authorized to work in the United States without sponsorship.

Nice To Haves

Healthcare data experience (e.g., Epic, HL7, FHIR, claims data) is strongly preferred
Experience with CI/CD, DevOps practices, and infrastructure-as-code tools is a plus

Responsibilities

Own the end-to-end architecture and delivery of scalable data solutions, with a strong emphasis on Databricks-based platforms and modern cloud ecosystems
Lead design and implementation of data pipelines, data models, and transformation frameworks that support analytics, reporting, and advanced use cases
Serve as a primary client-facing technical leader, building trusted relationships and guiding stakeholders through complex data challenges and solution decisions
Translate ambiguous business requirements into clear, actionable technical architectures and delivery plans
Establish and enforce best practices across data engineering, including data ingestion (pipeline orchestration, testing, and optimization) and DevOps.
Drive platform strategy and architecture decisions, including lakehouse design, medallion architecture, and governance frameworks
Lead and mentor delivery teams, providing technical guidance, code reviews, and hands-on support to ensure successful project outcomes
Collaborate with cross-functional teams including data architects, analysts, and client stakeholders to ensure alignment and value delivery
Identify risks, proactively address challenges, and ensure high-quality, timely delivery across engagements
Contribute to internal capability building, including reusable frameworks, accelerators, and thought leadership
Support business development efforts by shaping technical solutions, contributing to proposals, and participating in client discussions