Databricks Solution Architect

EXL • San Francisco, CA

1d

About The Position

The Solution Architect will own the end-to-end design of the organization’s modern data platform on Databricks. The role translates business needs across Risk, Fraud, Finance, Marketing, Collections, and Product into scalable, secure data architectures that integrate with existing systems. The Solution Architect owns architectural decisions across teams and partners closely with the Principal Engineer, who develops detailed designs and working pipelines.

The role defines target-state enterprise data architecture across ingestion, medallion layering, serving, and machine learning, including connections to source systems, the legacy warehouse, loan servicing platforms, and vendor feeds. It also leads discovery, solution designs, reference architectures, delivery roadmaps, migration planning, security and governance decisions, cost and capacity modeling, architecture reviews, and vendor evaluations. The posting additionally describes overseeing organizational data engineering solutions, leading high-impact initiatives, ensuring alignment with organizational goals, and providing executive guidance.

Requirements

• 12+ years of experience in data engineering or architecture, including 5+ years on the Databricks lakehouse platform.
• Strong Spark, Delta Lake, and SQL skills, plus data modeling experience with medallion and dimensional approaches.
• AWS architecture experience covering networking, IAM, encryption, and multi-account design.
• Experience designing secure platforms for PII/NPI, including Unity Catalog access models and lineage.
• Experience planning migrations from legacy Hadoop, Hive, and warehouse systems.
• Ability to choose between batch and streaming and design for both, including Kafka and Structured Streaming.
• Experience with disaster recovery and high-availability design.
• FinOps and Databricks cost and capacity modeling experience, including development of a business case.
• Stakeholder management, vendor management, and clear architecture documentation skills.
• Graduate in Computer Science, Data Science, or a related field.
• The qualifications section also states 6–8 years of experience in data engineering or a related field.

Nice To Haves

• Banking, lending, or other regulated financial services experience.
• Marketing data architecture experience, including customer 360, consent, and campaign measurement.
• Collections data architecture experience, including delinquency, agency, and dialer feeds.
• ML/MLOps architecture on Databricks, including feature store and MLflow.
• Databricks and AWS Solution Architect certifications.

Responsibilities

• Define the enterprise Databricks data architecture for ingestion, medallion layering, serving, and ML, and its integration with source systems, the legacy warehouse, loan servicing platforms, and vendor feeds.
• Run discovery with business and technology stakeholders and translate requirements into solution designs, reference architectures, and delivery roadmaps.
• Design marketing data architecture for identity resolution, segmentation, campaign and attribution measurement, and compliant data sharing with platforms and vendors.
• Design collections data architecture for near-real-time delinquency and contact data, agency and vendor file exchange, and compliant handling of contact preferences and regulatory restrictions.
• Make cross-cutting decisions on batch versus streaming, build versus buy, workspace topology, network and multi-account design, and disaster recovery.
• Design security and compliance architecture with InfoSec, including Unity Catalog access models, PII/NPI protection, encryption, audit logging, and lineage.
• Plan and sequence migration of legacy Hadoop, Hive, EMR, and warehouse workloads, including cutover strategy and risk mitigation.
• Build cost and capacity models, including DBU forecasting and FinOps guardrails, and develop business cases for platform investments.
• Chair the architecture review board and maintain standards, reference patterns, and decision records.
• Manage relationships with the Databricks account team and cloud provider, and lead vendor evaluations and proofs of concept.
• Oversee organizational data engineering solutions, lead high-impact initiatives, ensure alignment with organizational goals, and provide executive guidance.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior