Senior Data Engineer - Databricks

Capgemini • Atlanta, GA

2d • $89,400 - $166,398

About The Position

Capgemini is seeking an experienced Data Engineer to design, build, and optimize scalable data solutions on the Databricks Lakehouse Platform. The role involves developing end-to-end ETL/ELT pipelines with Apache Spark (PySpark), Delta Lake, and cloud platforms such as Azure, AWS, or GCP; data modeling and warehousing; performance tuning; and integration of structured and unstructured data from multiple sources. The engineer will define and deploy end-to-end data architecture, including Databricks medallion architecture, lead large engineering and development teams delivering Databricks solutions, and manage senior client stakeholders.

The role requires substantial experience with Databricks, PySpark/Python, SQL, cloud platforms, Delta Lake, and large-scale data. It also involves capabilities across Unity Catalog, Delta Sharing and Marketplace, Databricks Apps, and Databricks Genie, AI/BI, and data visualization. Familiarity with CI/CD and orchestration tools such as Airflow is desired, as is experience with ETL tools such as Informatica, Talend, or Matillion. The posted base compensation range is $89,400–$166,398. Eligible regular, full-time employees may receive benefits according to local policy and eligibility.

Requirements

• At least 6 years of experience in Data Engineering.
• At least 4 years of experience designing and delivering data engineering solutions with Databricks.
• Expert-level hands-on development knowledge in PySpark/Python.
• Expert-level hands-on SQL development skills.
• Hands-on experience with at least two of Azure, AWS, and Google Cloud.
• Prior expertise deploying Delta Lake solutions into production.
• Experience extracting value from large and complex datasets from various sources and databases.
• Solid grasp of database engineering and design principles.
• Strong knowledge of Unity Catalog.
• Expertise in Delta Sharing and Marketplace.
• Experience working with Databricks Apps.
• Working knowledge of Databricks Genie, AI/BI, and Databricks data visualization capabilities.

Nice To Haves

• Familiarity with CI/CD methods.
• Background working with orchestration tools such as Airflow.
• Past or current experience with ETL tools such as Informatica, Talend, or Matillion.
• Conceptual, logical, and physical data modeling experience.

Responsibilities

• Define and deploy end-to-end data architecture, including Databricks medallion architecture.
• Lead large teams of engineers and developers to deploy Databricks solutions.
• Manage senior client stakeholders.
• Design, build, and optimize scalable data solutions on the Databricks Lakehouse Platform.
• Develop end-to-end ETL/ELT pipelines using Apache Spark (PySpark), Delta Lake, and cloud platforms.
• Perform data modeling, data warehousing, and performance tuning.
• Integrate structured and unstructured data from multiple sources.

Benefits

• Paid time off, including vacation of 12–25 days depending on employee grade, company-paid holidays, personal days, and sick leave, subject to policy.
• Medical, dental, and vision coverage, or provincial healthcare coordination in Canada.
• Retirement savings plans, such as 401(k) in the U.S. or RRSP in Canada.
• Life and disability insurance.
• Employee assistance programs.
• Other benefits as provided by local policy and eligibility.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed