Senior Databricks SME

BuzzClan LLC • Dallas, TX

4d

About The Position

The Senior Databricks SME leads the design and delivery of enterprise Databricks Lakehouse solutions and advises clients on data platform strategy and Databricks adoption. The role covers Lakehouse architecture, data engineering, machine learning and MLOps, client engagement leadership, and development of the firm's Databricks practice. The position is based in Dallas, Texas, and is listed as full-time/contract.

Requirements

• 10+ years of professional experience in data engineering, analytics, or platform architecture.
• 5+ years of deep, hands-on Databricks experience, including enterprise-scale production deployments.
• Expert-level proficiency in Apache Spark (PySpark and/or Scala), Delta Lake, and SQL.
• A proven track record designing multi-cloud Lakehouse architectures across Azure, AWS, or GCP.
• Strong experience with Unity Catalog for data governance, access control, and lineage tracking.
• Demonstrated ability to lead client-facing engagements and communicate complex technical concepts to executive audiences.
• Experience in consulting or professional services, including managing multiple concurrent engagements.
• Bachelor's degree in Computer Science, Engineering, Data Science, or a related field, or equivalent practical experience.

Nice To Haves

• Databricks Certified Data Engineer Professional or Machine Learning Professional certification.
• Experience with Delta Live Tables, Databricks Workflows, and Photon engine optimization.
• Familiarity with complementary tools such as dbt, Terraform/Pulumi (IaC), and CI/CD pipelines for data platforms.
• Hands-on production experience with MLflow, Feature Store, and Model Serving.
• Prior experience as a Databricks partner or in a Databricks-aligned consulting practice.
• Advanced degree (M.S. or Ph.D.) in a quantitative or technical discipline.

Responsibilities

• Lead end-to-end design of Databricks Lakehouse architectures, including Delta Lake, Unity Catalog, and bronze/silver/gold medallion data models.
• Define reference architectures, best practices, and reusable accelerators for client engagements.
• Evaluate and recommend deployment patterns across Azure Databricks, AWS Databricks, and GCP according to client infrastructure and governance needs.
• Architect and oversee large-scale ETL/ELT pipelines using Apache Spark, Delta Live Tables, and Databricks Workflows.
• Design real-time and batch ingestion frameworks for structured, semi-structured, and unstructured data sources.
• Establish data quality, lineage, and observability standards using Unity Catalog, Delta Lake features, and complementary tooling.
• Guide clients in implementing end-to-end ML workflows using MLflow, Feature Store, and Model Serving on Databricks.
• Advise on MLOps patterns, including model versioning, A/B testing, monitoring, and automated retraining pipelines.
• Collaborate with data science teams to productionize models and integrate ML outputs into business applications.
• Advise senior client stakeholders on data platform strategy and Databricks adoption.
• Lead discovery workshops, architecture reviews, and technical assessments to define solution roadmaps.
• Support pre-sales efforts by developing proposals, estimating level of effort, and delivering technical demonstrations.
• Develop Databricks-related frameworks, white papers, and training curricula as intellectual property for the firm.
• Mentor and upskill consultants across the data practice and foster a community of Databricks practitioners.
• Represent the firm at industry events, Databricks partner summits, and webinars.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior