Data Architect

Subway•Shelton, CT

About The Position

The Data Architect is a senior technical authority responsible for defining and evolving the enterprise data platform architecture built on Databricks. This role leads the design of lakehouse, streaming, and batch data solutions that power analytics, reporting, and AI/ML use cases across the business, and guides the modernization of legacy data warehouse workloads onto a modern Databricks lakehouse. The Senior Data Architect partners closely with Data Engineering, Analytics, Platform, Security, and Business teams to ensure data solutions are scalable, secure, cost-efficient, and aligned with enterprise architecture standards, while mentoring engineers on Databricks best practices.

Requirements

Deep expertise with the Databricks platform and ecosystem — Delta Lake, Unity Catalog, MLflow, Databricks Workflows, and Delta Live Tables.
Strong understanding of modern data architectures: lakehouse, data lake, data warehouse, and data mesh concepts.
Expert-level proficiency in SQL and Python/PySpark; working knowledge of Scala is a plus.
Experience with distributed data processing frameworks (e.g., Apache Spark) at enterprise scale.
Experience with cloud platforms (AWS, Azure, or GCP) and native data services.
Proficiency with orchestration tools such as Databricks Workflows, Airflow, or Azure Data Factory.
Experience with data governance, security, and access-control frameworks (RBAC/ABAC).
Experience with data-quality and observability tooling (e.g., Great Expectations, Monte Carlo, Databricks Lakehouse Monitoring).
Working knowledge of Infrastructure-as-Code (Terraform, Pulumi, or ARM/Bicep).
Ability to translate business requirements into scalable, cost-efficient technical designs.
Strong communication skills and ability to influence architecture decisions across engineering and business teams.
Bachelor's degree in Computer Science, Engineering, Data, Information Systems, or a related field (or equivalent practical experience).
8–12+ years of experience in data engineering, data architecture, or platform engineering.
Demonstrated experience architecting production-grade data platforms on Databricks, Snowflake, and/or Amazon Redshift.
Experience operating cloud-based, distributed data platforms at enterprise scale.

Nice To Haves

Databricks Certified Data Engineer Professional or Databricks Certified Data/AI Solutions Architect.
Experience leading large-scale data platform migrations (e.g., Redshift to Databricks, on-prem to cloud).
Exposure to ML/AI platform engineering — feature stores, model serving, and MLOps integration.
Familiarity with Microsoft Purview, Collibra, or Alation for enterprise data governance.
Advanced degree (Master's) in Computer Science, Engineering, or a related field.
Experience in QSR, Retail, CPG, or Franchise industry environments.

Responsibilities

Define and lead enterprise architecture for data platforms built on Databricks — including lakehouse, streaming, and batch architectures; architect Medallion (Bronze / Silver / Gold) pipeline patterns and design self-service capabilities leveraging Unity Catalog, Delta Lake, and Delta/Iceberg interoperability for domain teams.
Define architecture and migration patterns for modernizing legacy data warehouse workloads (e.g., Redshift, Snowflake) onto the Databricks lakehouse; establish architecture standards, design patterns, and technical guardrails across the data engineering organization.
Partner with Data Engineering teams to implement robust, reusable frameworks and pipeline orchestration; guide adoption of Databricks features — Delta Live Tables, Unity Catalog, MLflow, Databricks Workflows — and establish monitoring, observability, and reliability standards for production data pipelines.
Define and enforce data governance practices — data quality, lineage, cataloging, and access controls using Unity Catalog; implement secure data access models (RBAC/ABAC) and champion metadata management and data-contract enforcement as core architecture practices.
Serve as the technical authority for data platform architecture, advising Analytics, BI, Data Science, and Product teams; lead architecture and design reviews for complex data initiatives; influence technology selection and long-term platform direction.
Mentor data engineers and architects on Databricks best practices and modern data architecture patterns; drive cost-optimization strategies for Databricks and cloud compute/storage; define and track platform KPIs — reliability, data freshness, SLA adherence, and DBU consumption efficiency.