Senior Data Engineer - Databricks

Capgemini • Atlanta, GA

21h

About The Position

Capgemini is seeking an experienced Data Engineer to design, build, and optimize scalable data solutions on the Databricks Lakehouse Platform. The role involves developing end-to-end ETL/ELT pipelines using Apache Spark (PySpark), Delta Lake, and cloud platforms such as Azure, AWS, or GCP, along with data modeling, data warehousing, performance tuning, and integrating structured and unstructured data from multiple sources.

The engineer will define and deploy end-to-end data architecture incorporating Databricks medallion architecture, lead large teams delivering Databricks solutions, and manage senior client stakeholders. The position calls for hands-on expertise with Databricks, PySpark/Python, SQL, cloud platforms, and production Delta Lake solutions.

Capgemini describes itself as a global business and technology transformation partner with 340,000 team members in more than 50 countries. It works with clients on digital and sustainable transformation, delivering services and solutions spanning strategy, design, engineering, AI, cloud, and data.

Requirements

• At least 6 years of experience in data engineering.
• At least 4 years of experience designing and delivering data engineering solutions with Databricks.
• Expert hands-on development knowledge in PySpark/Python.
• Expert hands-on SQL development skills.
• Hands-on experience with at least two of the following cloud platforms: Azure, AWS, and Google Cloud.
• Prior expertise deploying Delta Lake solutions into production.
• Experience extracting value from large and complex data sets from various sources and databases.
• Solid grasp of database engineering and design principles.
• Strong knowledge of Unity Catalog.
• Experience with Delta Sharing and Marketplace.
• Experience working with Databricks Apps.
• Working knowledge of Databricks Genie, AI/BI, and Databricks Data Visualization capabilities.

Nice To Haves

• Familiarity with CI/CD methods.
• Background working with orchestration tools such as Airflow.
• Past or current experience with ETL tools such as Informatica, Talend, or Matillion.
• Experience with conceptual, logical, and physical data modeling.

Responsibilities

• Define and deploy end-to-end data architecture that includes Databricks medallion architecture.
• Lead large teams of engineers and developers to deploy Databricks solutions.
• Manage senior client stakeholders.
• Design, build, and optimize scalable data solutions on the Databricks Lakehouse Platform.
• Develop end-to-end ETL/ELT pipelines using Apache Spark (PySpark), Delta Lake, and cloud platforms.
• Integrate structured and unstructured data from multiple sources.

Benefits

• Paid time off, including vacation of 12–25 days depending on employee grade, company-paid holidays, personal days, and sick leave.
• Medical, dental, and vision coverage, or provincial healthcare coordination in Canada.
• Retirement savings plans, such as a 401(k) in the U.S. or RRSP in Canada.
• Life and disability insurance.
• Employee assistance programs.
• Other benefits as provided by local policy and eligibility.
• The role may be eligible for variable incentives, bonuses, or commissions, depending on position and applicable laws.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed