Databricks Solution Architect

EXL • San Francisco, CA

23h

About The Position

The Solution Architect owns the end-to-end design of the organization’s modern data platform on Databricks. The role translates business needs across Risk, Fraud, Finance, Marketing, Collections, and Product into scalable, secure data architectures that integrate with existing systems. The architect owns cross-team architectural decisions and works with the Principal Engineer, who develops detailed designs and working pipelines.

The role defines the target-state enterprise data architecture, leads discovery and roadmap development, and designs architecture for marketing and collections data needs. It also makes cross-cutting platform decisions, develops security and compliance architecture with InfoSec, plans legacy workload migrations, builds cost and capacity models and business cases, chairs the architecture review board, and manages relationships with Databricks and cloud-provider vendors.

The posting also describes broader responsibilities to oversee organizational data engineering solutions, lead high-impact initiatives, align work with organizational goals, and provide executive guidance. A separate qualifications section asks for a graduate in Computer Science, Data Science, or a related field and 6–8 years of experience in data engineering or a related field; this differs from the must-have skills section, which requires 12+ years in data engineering or architecture, including 5+ years on Databricks.

Requirements

• 12+ years of experience in data engineering or architecture, including 5+ years on the Databricks lakehouse platform (must-have skills section).
• Strong Spark, Delta Lake, and SQL skills, plus data modeling experience with medallion and dimensional models.
• AWS architecture experience covering networking, IAM, encryption, and multi-account design.
• Experience designing secure platforms for PII/NPI, including Unity Catalog access models and lineage.
• Experience planning migrations from legacy Hadoop/Hive/warehouse systems.
• Ability to choose between batch and streaming approaches and design for both, including Kafka and Structured Streaming.
• Experience with disaster-recovery and high-availability design.
• Experience with FinOps and Databricks cost and capacity modeling, including development of a business case.
• Stakeholder-management and vendor-management skills, and ability to produce clear architecture documentation.
• Graduate in Computer Science, Data Science, or a related field (qualifications section).
• 6–8 years of experience in data engineering or a related field (qualifications section; differs from the must-have experience requirement).

Nice To Haves

• Banking, lending, or other regulated financial-services experience.
• Marketing data architecture experience, including customer 360, consent, and campaign measurement.
• Collections data architecture experience, including delinquency, agency, and dialer feeds.
• ML/MLOps architecture on Databricks, including feature store and MLflow.
• Databricks and AWS Solution Architect certifications.

Responsibilities

• Define the enterprise Databricks data-platform architecture, including ingestion, medallion layering, serving, ML, and integration with source systems, the legacy warehouse, loan servicing platforms, and vendor feeds.
• Conduct discovery with business and technology stakeholders and turn requirements into solution designs, reference architectures, and delivery roadmaps.
• Design marketing architecture for identity resolution, segmentation, campaign and attribution measurement, and compliant data sharing with marketing platforms and vendors.
• Design collections architecture for near-real-time delinquency and contact data, agency and vendor file exchange, and compliant handling of contact preferences and regulatory restrictions.
• Make cross-cutting architectural decisions, including batch versus streaming, build versus buy, workspace topology, network and multi-account design, and disaster recovery.
• Design security and compliance architecture with InfoSec, including Unity Catalog access models, PII/NPI protection, encryption, audit logging, and lineage.
• Plan and sequence migration of legacy Hadoop/Hive/EMR and warehouse workloads, including cutover strategy and risk mitigation.
• Build cost and capacity models, including DBU forecasting and FinOps guardrails, and develop business cases for platform investments.
• Chair the architecture review board and maintain standards, reference patterns, and decision records.
• Manage relationships with the Databricks account team and cloud provider; lead vendor evaluations and proofs of concept.
• Oversee organizational data engineering solutions.
• Lead high-impact initiatives and ensure alignment with organizational goals.
• Provide executive guidance.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior