Databricks Architect

Cynet Systems•Princeton, NJ

About The Position

The Databricks Architect will design and build governed, conversational self-service on Databricks, incorporating Genie, Unity Catalog metric views, a semantic and context layer, and multi-agent orchestration. The role owns the end-to-end architecture, from data sources to the answers presented to business users, and designs a federated model in which a central hub shares definitions while business units or entities retain their own data and context. The architect defines the semantic and context layer, including ontology, common keys, governed metric views, business rules, entitlements, certification status, lineage, and knowledge sources. They establish how local definitions and metric views are published to the central hub, mapped to a common model, and reused without copying data. They also establish governance for metric changes and certification, including ownership, versioning, impact analysis, approval, and effective dating. The role combines hands-on engineering with explaining design choices to business and technology leaders and guiding a small team of engineers.

Requirements

10 or more years of experience in data engineering and data architecture, including at least 3 years as a hands-on architect on a delivery team.
At least 3 years of production experience with Databricks, including Unity Catalog governance, lineage and ABAC; Delta Lake; Databricks SQL; Lakehouse Federation; and Delta Sharing.
Proven experience building a governed semantic or metrics layer, such as Databricks metric views or dbt metrics, used by multiple teams.
Experience delivering at least one Genie, text-to-SQL, or LLM agent solution beyond a demo, including evaluation of answer accuracy.
Strong engineering proficiency in SQL, Python, PySpark, Git, and CI/CD for data assets.
Solid data modeling background, including conformed dimensions, common keys across sources, and reconciliation of figures to a system of record.
Demonstrated governance expertise, including access control, lineage, data sharing, and versioning of definitions in a multi-team or multi-entity setup.
Strong verbal and written communication skills, including the ability to explain design choices to business and technology leaders and facilitate workshops.

Responsibilities

Own the end-to-end architecture for governed, conversational self-service on Databricks, from data sources to the answer business users see.
Design a federated architecture with a central hub and shared definitions while allowing business units or entities to retain their own data and context.
Define the semantic and context layer, including ontology, common keys, governed metric views, business rules, entitlements, certification status, lineage, and knowledge sources.
Design methods for publishing local definitions and metric views to the central hub, mapping them to the common model and enabling reuse without copying data.
Establish governance for metric changes and certification, including ownership, versioning, impact analysis, approval, and effective dating.
Explain design choices to business and technology leaders and guide a small team of engineers.
Build Unity Catalog metric views, SQL functions for business rules, and ABAC policies with row filters and column masks.
Configure Genie spaces, Genie Agents, and Genie One, including instructions, example SQL, trusted assets, and benchmark questions.
Build Agent Bricks agents that route questions to the appropriate agent.
Set up Lakehouse Federation, Delta Sharing, and clean rooms where data must remain in place.
Maintain metric views and agent configuration in Git using CI/CD with Databricks Asset Bundles.
Measure answer accuracy using MLflow tracing and evaluation.