Data Platform Engineer

Lantern•Dallas, TX

1d•Hybrid

About The Position

Lantern is seeking a Data Platform Engineer to play a core role in shifting their data organization from a centralized model to one where domain teams build and own their own data products. This role will focus on building and administering the self-serve infrastructure, governance automation, and access controls that domain teams will utilize. It's a hands-on administration and enablement role, focusing on administering the Databricks platform and the Lakebase (managed Postgres) estate, enforcing standards through platform controls, automation, and review across Azure infrastructure, IAM, CI/CD, observability, and governance. The role also serves as the primary point of contact for domain teams, collaborating with Cloud Platform, Azure SQL DBA, and Security teams on shared foundations.

Requirements

A minimum of 4 years in a platform or infrastructure engineering role supporting a data or analytics organization.
Deep hands-on experience with Databricks (workspace administration, Unity Catalog, Delta Lake) and Azure (Entra ID, networking, Key Vault, Infrastructure-as-Code via Terraform or Bicep).
Hands-on PostgreSQL administration experience, including user and role management, connection configuration, query and index performance tuning, backup and restore, point-in-time recovery, and HA/DR.
Real experience with identity and access architecture (service principals vs. user-authorization (OBO) patterns).
Practical experience building and maintaining CI/CD pipelines with GitHub Actions and/or Azure DevOps.
Proficiency with Python or another scripting language for platform automation and tooling.
Experience using AI tools (e.g., GitHub Copilot, ChatGPT, Claude) to accelerate engineering work.
Comfort owning the infrastructure lifecycle: provisioning, decommissioning, and disaster recovery.
Experience working in a regulated environment, ideally healthcare, with HIPAA and/or SOC 2 requirements.
Comfort operating with elevated privileges in a regulated, PHI-bearing environment, and the judgment to escalate rather than grant exceptions unilaterally.
Ability to hold a standard without owning it — enforcing architecture and security standards set elsewhere in the organization through influence and automation rather than authority.
Strong problem-solving and troubleshooting ability, including under incident conditions.
Excellent communication and collaboration skills, and the ability to work effectively with cross-functional engineering teams.

Nice To Haves

Exposure to data mesh or domain-oriented data architecture, and the instinct that platform teams exist to make other teams self-sufficient rather than being the bottleneck.
Hands-on experience with Databricks Lakebase or another cloud-native Postgres service (Azure Database for PostgreSQL, Neon, Aurora), and with Postgres tooling such as psql, pgAdmin, and migration frameworks.
Familiarity with Azure SQL Database to coordinate effectively with the DBA team on integration points and workload placement, and experience integrating OLTP systems with a lakehouse.
Healthcare data environment experience (HIPAA-driven access controls, PHI segregation), and demonstrate the ability to learn new business models and technologies.
Experience with observability tooling such as Datadog or OpenTelemetry-based pipelines.
Experience with Databricks Asset Bundles, DLT, or dbt in a multi-team deployment model.
Relevant certifications (e.g., Databricks Data Engineer Professional, Azure Administrator, etc.).

Responsibilities

Administer the Azure Databricks and Unity Catalog platform against architecture standards, including workspace and metastore configuration, catalog and schema structure, cluster policies, and compute cost management.
Build the self-serve infrastructure layer, including provisioning workflows, templates, and access patterns for domain teams.
Act as the DBA for the platform’s Postgres estate, managing provisioning, user/role management, connection configuration, performance tuning, upgrades, backup/recovery, and incident response.
Maintain the sync patterns between Lakebase and the lakehouse.
Build and maintain Infrastructure-as-Code for the platform using Terraform against Azure.
Implement and enforce the platform’s approved access model across the lakehouse and Postgres layer.
Build the audit and lineage layer for decentralized ownership, including audit logging, lineage capture, sensitivity classification tagging, and custom event-sourced audit layers.
Operate platform observability across Databricks, Lakebase, and supporting Azure services.
Execute the infrastructure lifecycle, including provisioning, decommissioning, backup/restore validation, and disaster recovery testing.
Support CI/CD for data workloads across domain teams.
Maintain the integration between the platform and GitHub / GitHub Actions.
Collaborate with the Cloud Platform team on shared Azure foundations.
Coordinate with the Azure SQL DBA team on integration points and shared practices.
Maintain liaison relationships with Security, InfoSec, and Core Apps on cross-team platform dependencies.
Engage early in new domain-team project kickoffs to advise on platform capabilities and standards.
Serve as the escalation point for domain engineers facing platform-related issues.
Produce platform KPI reporting and contribute to the platform-as-product roadmap and backlog.
Capture reusable patterns in a shared knowledge base for domain teams.