Senior Data Lakehouse Architect (Databricks), Vice President

State Street•Boston, MA

1d•$120,000 - $202,500

About The Position

We are seeking a Senior Data Lakehouse Architect to design and lead the build-out of a Legal Data Lakehouse platform on AWS and Databricks. This role will drive the architecture, engineering, and governance of scalable, secure, and compliant data capabilities supporting legal operations, contract intelligence, eDiscovery, and AI/ML use cases. The ideal candidate brings deep expertise in Databricks, AWS data platforms, and enterprise data architecture, with experience delivering solutions in regulated environments aligned to security, compliance, and audit requirements. State Street’s Legal function operates across a broad set of contracts, matters, regulatory obligations, documents, and workflows that are distributed across multiple systems and formats. Building a modern Legal Data Lakehouse is critical to creating a trusted, governed foundation that brings these data sources together—making legal information easier to access, analyze, and use at scale. This role is critical to establishing a secure and scalable data foundation that enables legal analytics and AI use cases while strengthening governance, auditability, and global consistency across Legal.

Requirements

10+ years of experience in data architecture, engineering, or analytics platforms
5+ years of hands-on experience with Databricks and Apache Spark
Strong experience with AWS-based data platforms
Expertise in data governance, security, and compliance in regulated environments
Experience working with unstructured data and NLP/document processing pipelines
Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related technical discipline
Strong hands-on experience with the Databricks Lakehouse platform, including Delta Lake, Unity Catalog, Workflows, and MLflow
Deep expertise in AWS data platform services, including S3, Glue, EMR, Lambda, Redshift, and IAM
Proven experience architecting and delivering enterprise-scale data lakehouse solutions on AWS using Databricks
Advanced proficiency in Apache Spark (PySpark/Scala), SQL, and Python
Strong understanding of data governance and security, including access controls, metadata management, and encryption (KMS, CMK/BYOK)
Experience building end-to-end data pipelines (batch and streaming) and supporting AI/ML workloads within a lakehouse architecture

Nice To Haves

Relevant certifications strongly preferred: Databricks Certified Data Engineer / Architect, AWS Certified Solutions Architect (Associate or Professional)
Experience in Legal, Compliance, Financial Services, or other regulated industries
Understanding of legal data constructs, including contracts, clauses, obligations, and matters
Experience supporting legal AI use cases (contract analytics, document summarization, compliance monitoring)
Experience handling sensitive data in highly regulated, audit-driven environments

Responsibilities

Define and implement the end-to-end Legal Data Lakehouse architecture using Databricks (Delta Lake, Unity Catalog, Workflows) on AWS
Design multi-layered data architecture (Bronze, Silver, Gold) to support: Contract metadata and document ingestion, Legal matter management data, eDiscovery datasets, External regulatory and compliance feeds
Establish scalable ingestion frameworks (batch and streaming) for structured and unstructured legal data (PDFs, contracts, emails)
Lead development of ETL/ELT pipelines using Databricks, Spark, and Python/SQL
Integrate with enterprise platforms, including: Contract lifecycle management systems, AI platforms and LLM pipelines, Document repositories and enterprise content systems
Design patterns for extracting structured data from unstructured legal documents and persisting into Delta Lake
Enable downstream integration with enterprise data platforms, analytics tools, and AI/ML pipelines
Implement data governance frameworks using Databricks Unity Catalog and AWS-native controls (IAM, KMS)
Establish: Fine-grained access controls (row/column-level security), Data lineage and auditability
Ensure compliance with: Data privacy regulations (e.g., GDPR), Internal security and audit requirements
Partner with IAM teams to integrate with enterprise identity providers (e.g., Entra ID / Azure AD)
Architect data models supporting: Contract analytics, clause extraction, and obligation tracking, Legal AI use cases (contract review, litigation insights, compliance monitoring, legal spend analytics)
Design search and retrieval architectures (RAG) for enterprise legal knowledge bases
Enable entity extraction and knowledge graph frameworks
Integrate with LLM/GenAI platforms to support capabilities such as document summarization, Q&A, and workflow automation
Establish CI/CD pipelines and infrastructure-as-code (Terraform, Git-based workflows)
Define standards for: Code quality and versioning, Environment promotion (Dev / QA / Prod)
Implement observability and alerting for platform health and reliability
Partner with Legal and Technology leadership to define platform roadmap and priorities
Provide architectural governance and design oversight
Mentor data engineers and platform teams
Translate business and legal requirements into scalable, enterprise-grade solutions
Operate within a federated data and platform model, collaborating across engineering, security, and domain teams

Benefits

our retirement savings plan (401K) with company match
insurance coverage including basic life, medical, dental, vision, long-term disability, and other optional additional coverages
paid-time off including vacation, sick leave, short term disability, and family care responsibilities
access to our Employee Assistance Program
incentive compensation including eligibility for annual performance-based awards (excluding certain sales roles subject to sales incentive plans)
eligibility for certain tax advantaged savings plans
inclusive development opportunities
flexible work-life support
paid volunteer days
vibrant employee networks