Senior Data Engineer

Capgemini•New York, NY

20d•$85,000 - $95,000•Onsite

About The Position

Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you’d like, where you’ll be supported and inspired by a collaborative community of colleagues around the world, and where you’ll be able to reimagine what’s possible. Join us and help the world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world.

Requirements

Hands on programming experience in python and PYSpark
Advanced SQL skills with query optimization techniques
Strong understanding of AWS and distributed computing paradigms
Basic understanding of IAM roles security and AWS infrastructure
Hands on experience to work with large scale data platforms in development of cloud solutions in AWS data platform with proven track record in driving business success
Understanding of Dev Ops Pipelines using Jenkins GitLab
Well versed with Project management tools and Agile Methodology
Sound knowledge of data governance principles and tools alationglue data quality mesh

Nice To Haves

Ability to develop data ingestion programs to ingest realtime data from LIVE sources using Apache Kafka Spark Streaming and related technologies
Understanding of DBT Data Build Tool for data transformation and modeling

Responsibilities

Design and develop data ingestion programs to process large data sets in Batch mode using Glue Lambda S3 redshift and data bricks
Develop data ingestion programs to ingest realtime data from LIVE sources using Apache Kafka Spark Streaming and related technologies
Data modeling and Data architecture concepts
Suggest solution architecture for diverse technology applications
Data transformation and modeling using DBT Data Build Tool