Senior Data Engineer

Capgemini • Brisbane, CA

8d • $130,000 - $150,000 • Hybrid

About The Position

Capgemini is seeking a data engineering professional to design, develop, and optimize scalable data ingestion and processing solutions using Databricks native capabilities. The role focuses on large-scale data integration, cloud-based data platforms, Databricks Native Data Loader, Lakeflow Connect, Delta Lake, and enterprise data migration from relational databases such as Oracle, MySQL, and SQL Server. The position leads the design and implementation of modern data pipelines, mentors engineering teams, and collaborates with business and technology stakeholders to deliver reliable, scalable, high-performing data solutions.

The position is based in San Francisco, California, with a hybrid arrangement requiring three days per week in the office. The posted base compensation range is $130,000–$150,000. Capgemini states that regular full-time employees may be eligible for benefits under local policy and eligibility, including paid time off, holidays, personal days, sick leave, medical, dental and vision coverage, retirement savings plans, life and disability insurance, and employee assistance programs.

Requirements

• Deep expertise in data engineering, large-scale data integration, and cloud-based data platforms.
• Hands-on experience with Databricks Native Data Loader, Lakeflow Connect, Delta Lake, and enterprise data migration from relational databases such as Oracle, MySQL, and SQL Server.
• Experience building data ingestion and processing solutions using Databricks native capabilities.
• Experience developing batch and streaming data pipelines and ELT/ETL pipelines.
• Experience with Spark, cloud-native technologies, and distributed processing technologies.
• Ability to lead technical design and architectural decisions for data solutions.
• Ability to mentor engineers and collaborate with technical and business stakeholders.

Responsibilities

• Design and implement enterprise-scale data ingestion frameworks using Databricks Native Data Loader, Lakeflow Connect, and other Databricks native capabilities.
• Build and optimize batch and streaming pipelines for data from Oracle, MySQL, SQL Server, APIs, and other enterprise sources.
• Develop ELT/ETL pipelines using Databricks Spark, Delta Lake, and cloud-native technologies.
• Implement end-to-end data movement, transformation, validation, and monitoring solutions.
• Lead architectural decisions concerning data ingestion, storage, processing, and governance.
• Create and maintain optimal data pipeline architecture.
• Build scalable pipelines that transform raw and unstructured data into analytics-ready datasets.
• Assemble large, complex datasets that meet functional and nonfunctional business requirements.
• Identify, design, and implement process improvements, including automation, infrastructure optimization, and scalability enhancements.
• Develop infrastructure for extracting, transforming, and delivering data from diverse sources using distributed processing technologies.
• Build and maintain integrations for data extraction, transformation, and loading.
• Implement data validation and quality-monitoring processes to ensure production data is accurate and reliable.
• Perform root cause analysis on data issues and recommend corrective actions.
• Write unit and integration tests, adopt Test-Driven Development practices, and maintain technical documentation.
• Ensure platform reliability, performance optimization, and adherence to enterprise standards.
• Provide technical leadership and mentorship to junior engineers.
• Collaborate with Product, Engineering, Analytics, Architecture, and business stakeholders to understand requirements and deliver data solutions.
• Drive best practices in coding, DevOps, CI/CD, testing, and documentation.
• Participate in architecture reviews, code reviews, and technical design discussions.
• Support Agile development processes and contribute to sprint planning and estimation activities.

Benefits

• Paid time off, including vacation of 12–25 days depending on employee grade, company-paid holidays, personal days, and sick leave.
• Medical, dental, and vision coverage, or provincial healthcare coordination in Canada.
• Retirement savings plans, such as a 401(k) in the U.S. or RRSP in Canada.
• Life and disability insurance.
• Employee assistance programs.
• Other benefits as provided by local policy and eligibility.
• Potential additional compensation, such as variable incentives, bonuses, or commissions, depending on the position and applicable laws.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed