Lead Data Architect / Data Engineer

eStaffLLC•Sacramento, CA
•Hybrid

About The Position

The Lead Data Architect/Data Engineer will work on a hybrid assignment for a Sacramento, California-based client, helping define architecture and deliver production-grade pipelines across multiple phases of a Databricks platform. The role combines hands-on data engineering and architecture with leadership, stakeholder engagement, governance collaboration, and support for other departments building pipelines. The position involves establishing platform architecture, naming conventions, standards, and technical documentation in partnership with the Program Manager and the Data Hub governance group. It also includes recommending protections for sensitive data, improving pipeline reliability, and publishing datasets and documentation to enable plain-language discovery and AI-enabled analysis. The role will create and maintain instruction files for AI coding assistants to promote consistent development standards.

Requirements

  • Strong leadership experience, including executive-level stakeholder engagement.
  • At least 7 years of experience in data engineering, data architecture, or closely related roles.
  • Proven experience delivering large-scale, enterprise data engineering or architecture initiatives.
  • Strong hands-on expertise in Python and SQL.
  • Experience with Apache Spark or cloud data platforms such as Databricks, Snowflake, BigQuery, or similar; Databricks is strongly preferred.
  • Experience integrating data from REST APIs, databases, and other source systems.
  • Excellent communication skills and strong writing and documentation practices.

Nice To Haves

  • Hands-on Databricks experience with Delta Lake, Unity Catalog, and Databricks Jobs.
  • Experience governing sensitive or confidential data, including access control, masking, and privacy requirements.
  • Experience operating within a data governance program, including stewardship, data contracts, and cross-agency sharing agreements.
  • Experience writing skill files, prompts, or instruction sets for AI coding assistants.
  • Experience with GIS or location-based datasets.

Responsibilities

  • Build and maintain automated ingestion and transformation pipelines that bring data from public portals and open data sources, REST APIs, and internal departmental databases and systems into Databricks Delta tables.
  • Partner with the Program Manager to establish and maintain platform architecture, naming conventions, standards, and technical documentation.
  • Collaborate with the Data Hub governance group on platform and data architecture, data-sharing patterns, data contracts and ownership, and data quality and reliability standards.
  • Design recommendations for protecting sensitive data, including row-level and column-level access controls and separation of confidential or internal data from public datasets.
  • Improve reliability through monitoring, alerting, data quality checks, and validation frameworks.
  • Publish datasets and documentation that support plain-language exploration and AI-enabled analysis tools.
  • Write and maintain skill files—instruction files for AI coding assistants—to enforce consistent pipeline patterns and development standards.
  • Support and enable other departments to build pipelines using shared templates and best practices.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service