Senior Data Engineer - Databricks

Capgemini • New York, NY

1d • $89,400 - $166,398 • Onsite

About The Position

Capgemini is hiring a Senior Data Engineer specializing in Databricks for an onsite role in New York City. The position leads the design and optimization of scalable data lakehouse solutions and oversees end-to-end project delivery, production support, stakeholder relationships, budgets, and service performance. The successful candidate serves as a technical leader and client-facing advisor, delivering reliable, high-quality data solutions efficiently and within contractual commitments.

The role develops and maintains architectures for structured and unstructured data, optimizes workflows for performance, scalability, and cost efficiency, and integrates Databricks with cloud services and enterprise systems. It also oversees support operations, service metrics, application automation and enhancements, reporting, cost analysis, and coordination with client and cross-functional teams.

The posted base compensation range is $89,400–$166,398. The position may also be eligible for variable incentives, bonuses, or commissions. Capgemini describes a comprehensive benefits package for regular, full-time employees; benefits vary by local policy and eligibility and may include paid time off and holidays, medical, dental, and vision coverage, retirement savings plans, life and disability insurance, and employee assistance programs.

Requirements

• At least 5 years of experience designing and implementing ETL/ELT pipelines using Databricks and Apache Spark.
• Strong proficiency in Python, SQL, and PySpark.
• At least 5 years of delivery and project management experience.
• Knowledge of Delta Lake, lakehouse concepts, and streaming data.
• Familiarity with CI/CD pipelines, Git version control, and DevOps practices.
• Understanding of data modeling, data warehousing, and performance tuning.
• Hands-on experience with AWS, Azure, or GCP.

Responsibilities

• Develop and maintain lakehouse architectures for structured and unstructured data.
• Optimize data workflows for performance, scalability, and cost efficiency.
• Collaborate with data scientists, analysts, and business stakeholders to deliver high-quality data solutions.
• Monitor and troubleshoot data pipelines to ensure reliability and accuracy.
• Integrate Databricks with AWS, Azure, or GCP cloud services and other enterprise systems.
• Oversee daily support operations and resolve client complaints within SLAs.
• Drive regular cross-platform training to build team product and system knowledge.
• Track support metrics, including ticket volume, resolution time, and CSAT; maintain SOPs and regular team rotations.
• Manage automation and enhancements for supported applications, build ThoughtSpot dashboards and custom reports, and transition new applications into support.
• Track project costs across GCP and Snowflake, propose cost-saving initiatives, and present monthly cost analyses to clients.
• Deliver monthly support summaries, proactively flag data issues, and coordinate with cross-functional teams including Adserver, Finance, Product, Yield, and Cadent.
• Manage end-to-end project delivery, production support, stakeholder relationships, budgets, and service performance.

Benefits

• Paid time off, including vacation (12–25 days depending on employee grade), company-paid holidays, personal days, and sick leave, subject to policy.
• Medical, dental, and vision coverage (or provincial healthcare coordination in Canada), subject to local policy and eligibility.
• Retirement savings plans, such as a 401(k) in the U.S. or RRSP in Canada.
• Life and disability insurance.
• Employee assistance programs.
• Potential variable incentives, bonuses, or commissions, depending on the position and applicable laws.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed