Senior Staff Engineer, Data Platform

Qualified Health PBC
•$215,000 - $235,000•Hybrid

About The Position

Qualified Health is hiring a Senior Staff Engineer to set technical direction for its Data Platform and build key components hands-on. The role centers on Databricks and ownership of the lakehouse architecture, data pipelines, governance, performance, and cost management. It also includes building and operating AI pipelines alongside the platform using Temporal for durable orchestration. The engineer will work across product, AI, and engineering teams to establish platform-wide strategy and architecture, anticipate future technical needs, shape the roadmap with leadership, and improve engineering effectiveness across the organization. The role combines broad technical leadership with hands-on implementation of complex systems, and includes defining how the team uses AI coding agents to deliver software effectively and safely.

Requirements

  • Bachelor's degree in Computer Science, Electrical Engineering, Mathematics, or a related field.
  • At least 12 years of software or data engineering experience.
  • Track record designing and operating production data platforms at scale and setting architecture across multiple teams.
  • Experience in a highly regulated domain such as healthcare, insurance, or finance.
  • Experience with a lakehouse stack, including Spark, Delta Lake, Unity Catalog, Workflows or equivalent, and performance and cost tuning at scale.
  • Experience building pipelines for ML or LLM applications, such as LLM-based extraction, embeddings and vector search, feature pipelines, or evaluation datasets.
  • Fluency in Python and SQL.
  • Sound software engineering practices, including testing, CI/CD, infrastructure as code, and code review.
  • Demonstrated practical use of agentic coding tools, such as Claude Code or similar, in real delivery work, and judgment about when to trust, verify, and constrain them.
  • Experience owning ambiguous, cross-team problems end to end and influencing without direct authority.
  • Clear written and verbal communication, including authoring design documents that others can build from.

Nice To Haves

  • Experience with healthcare data such as claims, EHR, FHIR, or HL7, and HIPAA-compliant handling of PHI.
  • Databricks certifications or experience with Delta Live Tables, Lakeflow, or Databricks Asset Bundles.
  • Experience with other orchestrators such as Airflow, Dagster, or Prefect, and a view on when each is appropriate.
  • Experience with cloud infrastructure (GCP, AWS, or Azure), Terraform, and Kubernetes.
  • Contributions to open-source data, workflow, or AI infrastructure projects.
  • Deep, hands-on expertise with Databricks; experience with Snowflake or Microsoft Fabric is also aligned.

Responsibilities

  • Architect and evolve the Databricks lakehouse, including medallion layering, Delta Lake, Unity Catalog governance, and workspace and compute strategy.
  • Design, build, and optimize Databricks Spark and SQL pipelines, including incremental processing, data quality enforcement, performance tuning, and cost management.
  • Define data contracts, quality checks, access controls, and PHI-aware governance in partnership with security and compliance.
  • Build data foundations for AI products, including LLM-ready datasets, retrieval and vector-store pipelines, evaluation data, and governed interfaces for AI and product teams.
  • Design durable AI workflows on Temporal, including scheduled and watermarked pulls from Databricks gold tables, file-ingestion workflows, and long-running LLM extraction and enrichment jobs.
  • Establish patterns for retries, idempotency, versioning, and backfills, and keep handoffs between Databricks and Temporal clean, observable, and well governed.
  • Use AI coding agents and define team practices for agentic development, including tooling, prompts, guardrails, code review, and testing.
  • Write and review technical designs, set platform-wide standards, and make long-term build-versus-buy decisions.
  • Mentor engineers across teams, unblock other pods, and communicate trade-offs to technical and non-technical stakeholders.
  • Set platform-wide strategy and architecture across multiple teams, anticipate future problems, shape the roadmap with leadership, and improve engineering effectiveness.

Benefits

  • Competitive salary with equity packages.
  • Medical, dental, and vision insurance.
  • Flexible working hours.
  • Hybrid work options.
  • Inclusive environment that fosters creativity and innovation.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service