Databricks Solution Architect - Onsite- W2

CLOUDSCOUTS SOFTWARE SOLUTIONS LLC • Upper Merion Township, PA

7d • Onsite

About The Position

We are looking for an experienced Databricks Solution Architect to design, architect, and lead the implementation of enterprise-scale data and AI solutions using the Databricks Data Intelligence Platform. The ideal candidate should have strong hands-on experience with Databricks, Lakehouse architecture, Apache Spark, Delta Lake, Unity Catalog, cloud platforms, data engineering, and modern data/AI architectures. The candidate will work closely with business stakeholders, data engineers, developers, cloud teams, and enterprise architects to translate business requirements into scalable, secure, high-performance Databricks solutions. Databricks' architecture guidance emphasizes Lakehouse architecture, governance, security, reliability, performance, cost optimization, and interoperability.

Requirements

• 12+ years of experience in Data Engineering, Data Architecture, Cloud Architecture, or related fields.
• 5+ years of strong Databricks experience, including architecture and implementation.
• Strong hands-on experience with: Databricks Lakehouse Platform, Apache Spark / PySpark, Delta Lake, Unity Catalog, Databricks SQL, Databricks Workflows / Jobs, Delta Live Tables / Lakeflow, Data governance and security.
• Strong understanding of Data Lake, Data Warehouse, and Lakehouse architectures.
• Experience designing enterprise-scale batch and real-time/streaming data pipelines.
• Strong programming experience with Python and/or Scala.
• Strong SQL development and performance-tuning skills.
• Experience with at least one major cloud platform: AWS, Microsoft Azure, or Google Cloud Platform.
• Experience with cloud storage technologies such as Amazon S3, Azure Data Lake Storage, or Google Cloud Storage.
• Experience with Terraform/IaC and CI/CD.
• Strong understanding of networking, IAM, authentication, encryption, and cloud security.
• Excellent communication, presentation, documentation, and stakeholder-management skills.

Nice To Haves

• Databricks certifications are highly preferred.
• Experience with MLflow, Model Serving, Mosaic AI, and Generative AI.
• Experience with LLM/RAG architectures and AI/ML platforms.
• Experience with Kafka or other event-streaming technologies.
• Experience with data integration tools such as Informatica, ADF, dbt, or equivalent.
• Experience with Power BI/Tableau and enterprise analytics architectures.
• Experience with enterprise data governance and compliance requirements.
• Experience leading architecture workshops and technical discussions with senior stakeholders.
• Strong understanding of data modeling, dimensional modeling, metadata management, data lineage, and master data concepts.

Responsibilities

• Design and implement enterprise-grade Databricks Lakehouse architectures across development, testing, and production environments.
• Define target-state architecture for data engineering, analytics, BI, machine learning, and AI workloads.
• Architect solutions using Databricks, Delta Lake, Apache Spark, Unity Catalog, Databricks SQL, Workflows, and Lakehouse Federation.
• Design and implement enterprise data governance, security, access control, lineage, and data-sharing strategies using Unity Catalog.
• Lead migration of legacy data platforms, data warehouses, and Hive Metastore environments to Databricks/Unity Catalog.
• Develop scalable data ingestion and processing architectures for batch and streaming workloads.
• Provide technical leadership for performance tuning, scalability, reliability, and cost optimization of Databricks environments.
• Design cloud-native solutions leveraging AWS, Azure, or GCP and integrate Databricks with cloud storage, networking, security, and identity services.
• Establish Infrastructure as Code (IaC) and deployment strategies using Terraform and CI/CD. Databricks currently recommends Terraform for automating workspace, networking, storage, and Unity Catalog infrastructure.
• Collaborate with DevOps teams to implement automated deployment pipelines and environment promotion.
• Provide technical guidance to data engineers and development teams and conduct architecture/design reviews.
• Work directly with customers and stakeholders to understand requirements and present architecture options and recommendations.
• Create architecture diagrams, technical design documents, standards, and implementation roadmaps.
• Troubleshoot complex production issues and provide architectural recommendations for resolution.
• Stay current with emerging Databricks Data + AI capabilities, Generative AI, ML, and Lakehouse technologies.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed