Azure Databricks Lead

Capgemini • Atlanta, GA

2d • $103,330 - $128,656 • Hybrid

About The Position

We are seeking a Azure Databricks Lead who is also hands-on in data engineering, with deep expertise in Azure Cloud, Snowflake, and Databricks. This role combines strategic architecture ownership with active implementation, covering data platform design, pipeline engineering, governance, performance optimization, and production support for enterprise-scale analytics.

Requirements

• 8+ years of experience in data engineering and data architecture, with strong hands-on delivery.
• Strong expertise in Azure data ecosystem, including ADLS, Azure Data Factory, Azure SQL/Synapse, Key Vault, Entra ID, and Monitor.
• Strong hands-on experience with Snowflake, including schema design, performance tuning, virtual warehouses, and role-based security.
• Strong hands-on experience with Databricks, including Delta Lake, Workflows, notebook orchestration, and Unity Catalog concepts.
• Advanced proficiency in Python, SQL, and Spark for large-scale data transformation.
• Strong communication and stakeholder management skills across technical and business audiences.

Responsibilities

• Define and implement end-to-end data architecture on Azure, Snowflake, and Databricks.
• Design scalable lakehouse and warehouse patterns for ingestion, transformation, and data serving.
• Build and maintain robust batch and near real-time data pipelines using Python, SQL, and Spark.
• Architect and implement medallion-style models and curated data layers for reporting and advanced analytics.
• Optimize cost and performance across Snowflake warehouses and Databricks compute/workloads.
• Establish monitoring, alerting, data quality checks, and operational reliability standards.
• Collaborate with business, product, analytics, and engineering teams to convert requirements into scalable solutions.
• Mentor team members and enforce architecture and engineering best practices.

Benefits

• Medical, dental, and vision coverage
• Retirement savings plans (e.g., 401(k) in the U.S., RRSP in Canada)
• Life and disability insurance
• Employee assistance programs
• Paid time off based on employee grade (A-F), defined by policy: Vacation: 12-25 days, depending on grade, Company paid holidays, Personal Days, Sick Leave

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed