Senior Data Engineer

Capgemini • San Francisco, CA

1d • Hybrid

About The Position

Capgemini is seeking a highly skilled data engineering professional to design, develop, and optimize scalable data ingestion and processing solutions using Databricks native capabilities. The role focuses on large-scale data integration, cloud-based data platforms, and enterprise migrations from relational databases such as Oracle, MySQL, and SQL Server. The engineer will lead the design and implementation of modern data pipelines, mentor engineering teams, and work with business and technology stakeholders to deliver reliable, scalable, high-performing data solutions.

The position is based in San Francisco, California, with a hybrid schedule requiring three days per week in the office. The posted base compensation range is $130,000–$150,000. The role may also be eligible for variable incentives, bonuses, or commissions. Capgemini offers a benefits package to regular, full-time employees, with availability subject to local policy and eligibility.

Requirements

• Deep expertise in data engineering, large-scale data integration, and cloud-based data platforms.
• Hands-on experience with Databricks Native Data Loader, Lakeflow Connect, and Delta Lake.
• Experience with enterprise data migration from relational databases such as Oracle, MySQL, and SQL Server.
• Experience building data ingestion and processing solutions with Databricks native capabilities.
• Experience developing batch and streaming pipelines and ELT/ETL solutions.
• Experience using Databricks, Spark, and cloud-native technologies.
• Experience with distributed data processing, data validation, data quality monitoring, testing, and production reliability.

Responsibilities

• Design and implement enterprise-scale data ingestion frameworks using Databricks Native Data Loader, Lakeflow Connect, and other Databricks native capabilities.
• Build and optimize batch and streaming pipelines that ingest data from Oracle, MySQL, SQL Server, APIs, and other enterprise sources.
• Develop ELT/ETL pipelines using Databricks, Spark, Delta Lake, and cloud-native technologies.
• Implement end-to-end data movement, transformation, validation, and monitoring solutions.
• Lead architectural decisions concerning data ingestion, storage, processing, and governance.
• Create and maintain optimal data pipeline architecture and scalable pipelines that turn raw and unstructured data into analytics-ready datasets.
• Assemble complex datasets to meet functional and nonfunctional business requirements.
• Identify, design, and implement process improvements, including automation, infrastructure optimization, and scalability enhancements.
• Develop infrastructure to extract, transform, and deliver data from diverse sources using distributed processing technologies.
• Build and maintain integrations for data extraction, transformation, and loading.
• Implement data validation and quality monitoring processes to ensure production data is accurate and reliable.
• Perform root cause analysis of data issues and recommend corrective actions.
• Write unit and integration tests, adopt Test-Driven Development practices, and maintain technical documentation.
• Ensure platform reliability, optimize performance, and adhere to enterprise standards.
• Provide technical leadership and mentoring to junior engineers.
• Collaborate with Product, Engineering, Analytics, Architecture, and Business stakeholders to understand requirements and deliver data solutions.
• Promote coding, DevOps, CI/CD, testing, and documentation best practices.
• Participate in architecture reviews, code reviews, and technical design discussions.
• Support Agile development processes and contribute to sprint planning and estimation.

Benefits

• Paid time off, including vacation of 12–25 days depending on employee grade, company-paid holidays, personal days, and sick leave, subject to policy.
• Medical, dental, and vision coverage, subject to local policy and eligibility.
• Retirement savings plans, such as a 401(k) in the U.S. or RRSP in Canada, subject to local policy and eligibility.
• Life and disability insurance, subject to local policy and eligibility.
• Employee assistance programs, subject to local policy and eligibility.
• Potential eligibility for variable incentives, bonuses, or commissions.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed