Databricks Data Engineering Consultant

SDG Group

2d • Hybrid

About The Position

SDG Group is hiring a Databricks Data Engineering Consultant to design, develop, and deliver modern data solutions for clients. As part of a consulting delivery team, the consultant will work with technical and business stakeholders to understand data requirements and build reliable, scalable solutions that support analytics, reporting, AI, and other data-driven initiatives. The role emphasizes hands-on Databricks experience, particularly Databricks SQL and SQL-based data engineering workloads, in complex enterprise data environments.

The consultant will develop and optimize transformations, validate data, support pipelines, and help ensure data is structured and delivered effectively for downstream business and analytical use cases. This role is suited to someone with a data engineering foundation who enjoys hands-on delivery, solving complex problems, working directly with clients, and expanding expertise across modern cloud and data platforms.

SDG Group is a global consulting firm focused on Data, Analytics, and AI. It describes its approach as combining industry knowledge, technical proficiency, and partnerships with technology providers. The company offers a collaborative environment, professional development, and exposure to modern technologies through client engagements and technology partnerships.

Requirements

• Approximately 2–5+ years of experience in data engineering, data warehousing, data integration, analytics engineering, or a related technical discipline.
• Professional hands-on Databricks experience, including demonstrated use of the platform for production data engineering and analytical workloads.
• Strong Databricks SQL experience, including writing, troubleshooting, and optimizing queries for transformation, validation, reconciliation, and downstream analytics.
• Strong SQL skills, including joins, CTEs, window functions, aggregations, and complex transformations.
• Working proficiency in Python or another relevant programming or scripting language.
• Strong understanding of data pipelines, ETL/ELT processes, data integration, data warehousing, and data transformation concepts.
• Understanding of dimensional modeling and modern data modeling concepts used in analytical environments.
• Experience working with modern cloud environments and enterprise-scale data platforms.
• Experience with modern data engineering, orchestration, or transformation tools.
• Experience validating, reconciling, and troubleshooting data across multiple source and target systems.
• Strong analytical and problem-solving skills, including the ability to troubleshoot data issues, understand complex requirements, and translate business needs into technical solutions.
• Strong written and verbal communication skills and ability to collaborate with clients, technical teams, and business stakeholders.
• Proactive, collaborative approach and willingness to continuously learn tools, technologies, and methodologies.

Nice To Haves

• Experience in consulting or another client-facing environment.
• Experience with Matillion and/or dbt.
• Experience with Databricks SQL Warehouses, Delta Lake or Delta Tables, Databricks Workflows and orchestration, Unity Catalog, and Databricks query and performance optimization.
• Experience with Python/PySpark and Snowflake.
• Experience supporting enterprise-scale data migration or modernization initiatives.
• Experience in pharmaceutical, life sciences, healthcare, or another highly regulated industry.

Responsibilities

• Design, build, test, and maintain scalable data solutions in Databricks, especially SQL-based transformation and data-processing workloads.
• Write, troubleshoot, and optimize complex SQL queries for transformation, validation, reconciliation, and analytical workloads.
• Build and maintain scalable data pipelines and processing solutions that support client analytics and business requirements.
• Integrate structured and unstructured data sources using modern ETL/ELT methodologies and data integration tools.
• Develop and maintain data models and warehouse structures for reporting, analytics, and downstream applications.
• Contribute to implementing and modernizing cloud-based data platforms using Databricks and related cloud technologies.
• Transform, validate, and manage enterprise data in accordance with development, testing, documentation, and deployment best practices.
• Identify, troubleshoot, and resolve data quality issues and improve reliability, scalability, and query performance.
• Work with clients and internal stakeholders to gather requirements, understand business challenges, communicate technical concepts, and deliver solutions aligned with client needs.
• Collaborate with data engineers, analysts, architects, data scientists, and business stakeholders throughout the project lifecycle.
• Stay current with Databricks capabilities, data engineering methodologies, and industry best practices, and contribute knowledge and ideas to the SDG community.

Benefits

• Competitive salary.
• 401(k) retirement plan with employer match.
• Medical, dental, vision, and disability coverage.
• 3+ weeks of paid vacation.
• Flexible holiday schedule.
• Hybrid working arrangements balancing on-site and remote work.
• Professional development and career advancement opportunities.
• Continuous training and exposure to modern Data, Analytics, and AI technologies.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed