Senior Databrick Data Engineering Consultant

SDG Group•Upper Merion Township, PA

3h•Hybrid

About The Position

As a Senior Databricks Data Engineering Consultant, you will lead the design, development, and delivery of modern data solutions for SDG Group clients. You will serve as a senior technical contributor and trusted partner on complex engagements, working with technical and business stakeholders to shape requirements, guide solution design, and deliver reliable, scalable platforms for analytics, reporting, AI, and other data-driven initiatives. The role requires extensive hands-on Databricks experience, particularly with Databricks SQL and SQL-based data engineering workloads. You will work in complex enterprise data environments, lead technical decisions, optimize transformations and pipelines, validate data, and establish scalable patterns and best practices. The position combines hands-on delivery with technical leadership, client partnership, mentoring, and ownership of complex engineering workstreams. SDG Group is a global consulting firm specializing in Data, Analytics, and AI. Its work combines industry expertise, technology proficiency, and strategic partnerships, and includes solutions across cloud, on-premises, hybrid, and sovereign environments.

Requirements

Approximately 5–7+ years of experience in data engineering, data warehousing, data integration, analytics engineering, or a related technical discipline, with demonstrated ownership of complex technical delivery.
Extensive hands-on professional Databricks experience, including production data engineering and analytical workloads.
Advanced Databricks SQL experience, including designing, troubleshooting, and optimizing complex queries and transformations for enterprise-scale workloads.
Advanced SQL skills, including joins, CTEs, window functions, aggregations, complex transformations, performance tuning, and data validation.
Strong working proficiency in Python or another relevant programming or scripting language.
Deep understanding of data pipelines, ETL/ELT processes, data integration, data warehousing, transformation patterns, and production data engineering best practices.
Strong understanding of dimensional modeling and modern data modeling concepts, with the ability to design scalable analytical data structures.
Strong experience with modern cloud environments and enterprise-scale data platforms, including architecture and modernization initiatives.
Experience with modern data engineering, orchestration, or transformation tools.
Demonstrated ability to lead data validation, reconciliation, troubleshooting, and performance optimization across multiple source and target systems.
Strong analytical and architectural judgment, including the ability to evaluate tradeoffs, solve ambiguous technical problems, and translate complex business requirements into scalable solutions.
Strong written and verbal communication skills, including the ability to lead technical discussions, influence stakeholders, explain complex concepts clearly, and build client credibility.
A proactive, collaborative approach, with demonstrated ability to mentor others, raise delivery standards, and expand technical and consulting expertise.

Nice To Haves

Consulting or other client-facing experience.
Experience with Matillion and/or dbt.
Experience with Databricks SQL Warehouses.
Experience with Delta Lake or Delta Tables.
Experience with Databricks Workflows and orchestration.
Experience with Unity Catalog and data governance.
Experience with Databricks query, cluster, and performance optimization.
Python or PySpark experience.
Snowflake experience.
Experience leading enterprise-scale data migration or modernization initiatives.
Experience in pharmaceutical, life sciences, healthcare, or another highly regulated industry.

Responsibilities

Lead the design, development, testing, and maintenance of scalable Databricks data solutions, especially SQL-based transformation and processing workloads.
Design, troubleshoot, and optimize complex SQL queries and transformation logic for data processing, validation, reconciliation, and analytics.
Own or lead data engineering workstreams, guide technical decisions, establish development approaches, and support high-quality delivery on complex client engagements.
Design and implement integrations across structured and unstructured data sources using modern ETL/ELT methodologies and data integration tools.
Design and review scalable data models and warehouse structures for reporting, analytics, AI, and downstream applications.
Lead or contribute to architecture and modernization of cloud-based data platforms using Databricks and related cloud technologies.
Drive technical delivery and promote strong practices for development, testing, documentation, deployment, maintainability, and governance.
Lead troubleshooting and optimization related to data quality, reliability, scalability, and query or pipeline performance.
Partner with client stakeholders to clarify requirements, explain tradeoffs, recommend solutions, manage technical expectations, and connect engineering decisions to business outcomes.
Collaborate with multidisciplinary teams, review technical work, share best practices, and mentor less experienced engineers and consultants.
Stay current with Databricks capabilities, data engineering methodologies, and industry best practices, and contribute technical knowledge to the SDG community.

Benefits

Competitive salary.
401(k) retirement plan with employer match.
Medical, dental, vision, and disability coverage.
3+ weeks of paid vacation.
Flexible holiday schedule.
Hybrid working arrangements balancing on-site and remote work.
Professional development and career advancement support.
Continuous training and exposure to modern Data, Analytics, and AI technologies.

Stand Out From the Crowd

Upload your resume and get instant feedback on how well it matches this job.

Upload and Match Resume