Solution Architect (Databricks Ecosystem)

Lumen Solutions Group Inc • Washington, DC

5d • Remote

About The Position

We are seeking a Senior IT Applications Solution Architect specializing in Databricks to drive the architectural vision, governance, and end-to-end design of our next-generation enterprise Lakehouse platform. In this strategic role, you will serve as the premier technical authority, bridging the gap between enterprise business strategies and technical execution. You will architect scalable, secure, and resilient application integrations, define framework standards, and collaborate across engineering and executive leadership teams to align our data applications with overarching organizational goals.

Requirements

• 8+ years of technical experience in data engineering and IT architecture
• At least 3+ years specifically operating as a Senior Solution Architect, Application Architect, or Principal Engineer.
• Deep, hands-on architectural experience designing and deploying enterprise-grade, high-availability production solutions natively on the Databricks platform.
• Proven track record of architecting complex Delta Live Tables (DLT) architectures and streaming applications within a large-scale corporate environment.
• Expert-level understanding of Medallion architecture, relational/dimensional data modeling, and master data management (MDM) paradigms.
• Strong experience establishing multi-environment CI/CD deployment pipelines for enterprise data platforms using tools like GitHub Actions, GitLab CI, Azure DevOps, or Terraform.
• Exceptional communication, facilitation, and negotiation skills with a demonstrated ability to steer cross-functional initiatives and influence technology roadmaps at the executive level.

Responsibilities

• Define and maintain the high-level architectural designs, integration patterns, and governance frameworks for the enterprise Databricks environment and its connected application ecosystem.
• Act as the primary conduit between enterprise application architects, data architects, and engineering teams to translate business capability demands into concrete, scalable technical solutions.
• Articulate complex architectural trade-offs, financial implications (FinOps), and platform capabilities to senior executives and non-technical business leaders.
• Establish rigorous coding standards, design patterns, and architectural review processes; provide senior-level technical mentorship to engineering squads.
• Design end-to-end integration patterns utilizing Databricks Delta Live Tables (DLT), structured streaming, and advanced orchestration workflows to power downstream business applications.
• Standardize the enterprise Medallion Architecture (Bronze, Silver, Gold layers) to optimize application performance, downstream system consumption, and data compliance.
• Architect optimization strategies for Spark jobs, cluster topologies, and data layouts using advanced features (e.g., Liquid Clustering, Z-Ordering, Photon engines) to maximize throughput and minimize compute spend.
• Define and implement robust, proactive platform monitoring, operational logging, and alerting frameworks across all cloud data workflows.
• Architect and govern automated deployment pipelines to manage Databricks assets, workspace configurations, and code repositories across multi-tier environments (Dev, QA, Prod).
• Collaborate with Cloud Infrastructure teams to design automated provisioning, environment scaling, and enterprise security policies (Unity Catalog governance, data masking, and access controls).

What This Job Offers

• Career Level: Senior
• Education Level: No Education Listed