Principal Consultant - Databricks

OZ Digital LLC

3d•Remote

About The Position

OZ is seeking a highly experienced Principal Consultant - Databricks to serve as the senior technical authority for complex enterprise data-platform engagements. The role owns the technical vision, target-state architecture, standards, and roadmaps for Azure Databricks engagements, from discovery through production delivery. It shapes secure, scalable Lakehouse solutions, guides multidisciplinary teams, supports pre-sales and delivery planning, and ensures platform decisions align with customer outcomes, operational needs, security, performance, reliability, and cost goals. This is a remote, U.S.-based role working primarily in the Eastern Time Zone. The consultant will work with customer stakeholders, Databricks, Microsoft, sales teams, and internal practices. OZ describes its culture as collaborative and focused on learning, innovation, client outcomes, and work-life balance. The company has more than 25 years of experience in Azure cloud solutions, Power Apps, intelligent automation, enterprise application integration, Azure data strategy, and artificial intelligence.

Requirements

At least 10 years of experience in enterprise data and analytics consulting.
At least 5 years architecting and delivering production Databricks or Lakehouse solutions.
At least 3 years in senior technical leadership, solution architecture, or principal-level consulting roles.
At least 3 years in an insurance firm in a consulting capacity.
Demonstrated experience serving as the senior technical authority for complex enterprise data-platform engagements, with accountability from discovery through production delivery.
Deep Azure Databricks expertise across platform architecture, data engineering, Unity Catalog governance, operationalization, performance, and cost management.
Strong command of Apache Spark, PySpark, Python, SQL, Delta Lake, batch and streaming patterns, ETL/ELT, data modeling, data quality, performance tuning, and production troubleshooting.
Working knowledge of Microsoft Entra ID, Azure networking and private connectivity, ADLS, secrets and key management, monitoring, and related Azure data services.
Experience with Git-based development, automated testing, CI/CD, environment promotion, infrastructure as code, deployment automation, observability, runbooks, and support models.
Strong consulting and business acumen, including the ability to connect architecture decisions to customer outcomes, delivery risk, scope, margin, adoption, and platform consumption.
Executive-level written, verbal, facilitation, and presentation skills, with the technical depth to engage architects and engineers.
Experience in discovery, estimation, staffing, proposal or SOW development, technical pre-sales, delivery leadership, and multidisciplinary team guidance.
Self-directed, collaborative leadership with sound judgment, resilience, empathy, and the ability to create structure in ambiguous situations.

Nice To Haves

Databricks Certified Data Engineer Professional certification is strongly preferred.
Azure Platform Architect Badge and/or Databricks Data + AI Platform Administrator accreditation.
Databricks Partner Champion or Delivery Expert recognition, partner or Center of Excellence leadership, co-sell experience, or work in regulated and security-sensitive industries.
Databricks Certified Generative AI Engineer Associate or Machine Learning Professional certification.
Experience with large-scale platform modernization, cloud transformation, data-warehouse migration, or consolidation involving Databricks.
Experience creating reusable offerings, accelerators, reference architectures, assessment frameworks, delivery playbooks, or intellectual property.
Experience overseeing production analytics, machine-learning, or AI workloads beyond proof-of-concept delivery.
Exposure to MLflow, model serving, Power BI, Microsoft Fabric, Azure Data Factory, and Microsoft Purview.

Responsibilities

Define target-state architecture, principles, standards, and roadmaps for complex Azure Databricks engagements.
Lead discovery and architecture workshops, translating business priorities, security needs, operational constraints, and success measures into actionable solution designs.
Architect secure and scalable enterprise Lakehouse solutions spanning ingestion, storage, transformation, orchestration, analytics, data sharing, and AI/ML enablement.
Establish engineering, governance, identity, networking, CI/CD, observability, reliability, performance, and cost-optimization standards.
Provide delivery oversight by reviewing critical implementation artifacts, identifying risks early, guiding technical decisions, and aligning delivered solutions with what was sold.
Support pre-sales and solutioning, including qualification, solution shaping, estimates, staffing plans, proposals, statements of work, demonstrations, and executive presentations.
Develop reusable consulting assets, including reference architectures, accelerators, assessment methods, estimation models, delivery playbooks, and thought leadership.
Mentor architects, engineers, and consultants, and collaborate with Databricks, Microsoft, customer stakeholders, sales teams, and internal practices.