Data Platform Engineer

Lantern•Dallas, TX

19h•Hybrid

About The Position

Lantern is the specialty care platform connecting people with the best care when they need it most. By curating a Network of Excellence comprised of the nation's top specialists for surgery, cancer care, infusions and more, Lantern delivers excellent care with significant cost savings to employers and their workforces. Lantern also pairs members with a dedicated care team, including Care Advocates and nurses, for the entirety of their care journey, helping them get back to good health, back to their families and back to work. With convenient access to specialists nationwide, Lantern means quality care is within driving distance for most. Lantern is trusted by the nation's largest employers to deliver care to more than 6 million members across the country. Learn more about us at lanterncare.com. Lantern’s data organization is moving from a centralized model — where the platform team owns most pipeline delivery — toward one where domain teams (Data Engineering, Data Ops, Analytics Engineering) build and own their own data products, and the platform team’s job is to make that possible. This role is core to that shift: building and administering the self-serve infrastructure, governance automation, and access controls that domain teams build on top of, rather than building domain pipelines directly. This is a hands-on administration and enablement role. Domain teams own their data products; the data architecture committee owns the standards. This role administers the Databricks platform and the Lakebase (managed Postgres) estate, holds the elevated privileges that go with that, and enforces our standards through platform controls, automation, and review — across Azure infrastructure and IAM, CI/CD, observability, and the governance layer of access control, audit, and lineage. It is also the platform’s front door for domain teams, from project kickoffs through day-to-day unblocking, and works in close partnership with the Cloud Platform, Azure SQL DBA, and Security teams on the shared foundations underneath.

Requirements

A minimum of 4 years in a platform or infrastructure engineering role supporting a data or analytics organization.
Deep hands-on experience with Databricks (workspace administration, Unity Catalog, Delta Lake) and Azure (Entra ID, networking, Key Vault, Infrastructure-as-Code via Terraform or Bicep).
Hands-on PostgreSQL administration experience, as this role is the DBA of record for the platform’s Postgres estate — user and role management, connection configuration, query and index performance tuning, backup and restore, point-in-time recovery, and HA/DR.
Real experience with identity and access architecture — service principals vs. user-authorization (OBO) patterns, and the tradeoffs between them.
Practical experience building and maintaining CI/CD pipelines with GitHub Actions and/or Azure DevOps.
Proficiency with Python or another scripting language for platform automation and tooling.
Experience using AI tools (e.g., GitHub Copilot, ChatGPT, Claude) to accelerate engineering work.
Comfort owning the infrastructure lifecycle: provisioning, decommissioning, and disaster recovery.
Experience working in a regulated environment, ideally healthcare, with HIPAA and/or SOC 2 requirements.
Comfort operating with elevated privileges in a regulated, PHI-bearing environment, and the judgment to escalate rather than grant exceptions unilaterally.
Ability to hold a standard without owning it — enforcing architecture and security standards set elsewhere in the organization through influence and automation rather than authority.
Strong problem-solving and troubleshooting ability, including under incident conditions.
Excellent communication and collaboration skills, and the ability to work effectively with cross-functional engineering teams.

Nice To Haves

Exposure to data mesh or domain-oriented data architecture, and the instinct that platform teams exist to make other teams self-sufficient rather than being the bottleneck.
Hands-on experience with Databricks Lakebase or another cloud-native Postgres service (Azure Database for PostgreSQL, Neon, Aurora), and with Postgres tooling such as psql, pgAdmin, and migration frameworks.
Enough familiarity with Azure SQL Database to coordinate effectively with the DBA team on integration points and workload placement, and experience integrating OLTP systems with a lakehouse.
Healthcare data environment experience (HIPAA-driven access controls, PHI segregation), and demonstrate the ability to learn new business models and technologies.
Experience with observability tooling such as Datadog or OpenTelemetry-based pipelines.
Experience with Databricks Asset Bundles, DLT, or dbt in a multi-team deployment model.
Relevant certifications (e.g., Databricks Data Engineer Professional, Azure Administrator, etc.).

Responsibilities

Administer the Azure Databricks and Unity Catalog platform against the architecture standards set by the data architecture committee — workspace and metastore configuration, catalog and schema structure, cluster policies, and compute cost management, including spend-by-team visibility and chargeback reporting. Feed back to the standards owners where a standard is blocking teams or no longer fits how the platform is being used.
Build the self-serve infrastructure layer — provisioning workflows, templates, and access patterns that let domain teams stand up and manage their own data products without depending on the platform team for every change.
Act as the DBA for the platform’s Postgres estate — provisioning and decommissioning of Lakebase instances, user and role management, connection configuration, query and index performance tuning, version upgrades, backup and point-in-time recovery, and database incident response. Maintain the sync patterns between Lakebase and the lakehouse.
Build and maintain Infrastructure-as-Code for the platform using Terraform against Azure — resource groups, networking and private endpoints, Entra ID app registrations, service principals, and Key Vault — so platform changes are reviewable, repeatable, and auditable.
Implement and enforce the platform’s approved access model across the lakehouse and the Postgres layer — Unity Catalog grants, OBO (on-behalf-of) authentication patterns, ABAC, row-level security, data masking, and role-based access across engineering, ops, and analytics personas.
Build the audit and lineage layer that makes decentralized ownership auditable — audit logging, lineage capture, sensitivity classification tagging, and the custom event-sourced audit layer needed where Databricks cannot natively capture app-layer events, including incident response when this layer breaks.
Operate platform observability across Databricks, Lakebase, and supporting Azure services (Datadog integration, metrics and log export, alerting, SLOs) so domain teams get reliability guarantees without building their own monitoring.
Execute the infrastructure lifecycle — provisioning, decommissioning, backup and restore validation, and disaster recovery testing for platform and database workloads.
There is no overnight on-call rotation for this role. Occasional availability outside core hours is expected to support scheduled deployments and major incidents.
Support CI/CD for data workloads across domain teams — deployment pipelines for notebooks, jobs, DLT pipelines, dbt projects, Databricks Asset Bundles (DABs), and Lakebase schema migrations.
Maintain the integration between the platform and GitHub / GitHub Actions — repo structure, branch policies, and CI/CD workflows that domain teams build their pipelines on top of.
Work with the Cloud Platform team on the shared Azure foundation — networking, Entra ID, Key Vault, policy, and cloud cost — with a clear boundary for where platform responsibility ends, and triage and hand off Azure infrastructure issues accordingly.
Coordinate with the Azure SQL DBA team, who own the Azure SQL estate independently — integration points between the two estates, shared practice on backup, HA/DR, and PHI handling, and input on workload placement.
Maintain a recurring liaison relationship with Security, InfoSec, and Core Apps on cross-team platform dependencies as they surface, and ensure platform capabilities meet HIPAA, SOC 2, and other regulatory requirements.
Engage early in new domain-team project kickoffs to advise on platform capabilities, access patterns, and the standards a design must meet, before teams build around constraints they did not know existed.
Serve as the escalation point for domain engineers stuck on cluster policies, permissions boundaries, database performance, or broken CI steps.
Produce platform KPI reporting (usage, uptime, deployment velocity, spend) and contribute to the platform-as-product roadmap and backlog.
Capture reusable patterns from kickoffs, incidents, and liaison work in a shared knowledge base so domain teams do not have to rediscover solutions.