AWS Senior Data Engineer

EXL•Jersey City, NJ

1d•$150,000 - $170,000

About The Position

The AWS Senior Data Engineer designs, develops, and maintains scalable data pipelines and architectures in AWS. The role collaborates with data scientists, analysts, and business stakeholders to deliver robust data solutions, and includes hands-on engineering as well as oversight of organizational data engineering solutions and leadership of high-impact initiatives.

Requirements

Bachelor’s degree in Computer Science, Software Engineering, MIS, or an equivalent combination of education and experience.
8 or more years of experience implementing and supporting AWS data lakes, data warehouses, and data applications for large enterprises.
Strong programming experience with Python, Shell scripting, and SQL.
Experience with AWS CloudFormation, S3, Athena, Glue, EMR/Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, and Secrets Manager.
Experience in serverless application development and data pipeline orchestration.
Experience analyzing, designing, developing, and implementing AWS data ingestion pipelines.
Knowledge of ETL/ELT, data modeling, and big data technologies.
Familiarity with data warehousing concepts and cloud-based architecture.
Strong problem-solving skills and attention to detail.
Excellent communication and teamwork abilities.
The qualifications section also states: graduate in Computer Science, Data Science, or a related field, with 6–8 years of experience in data engineering or a related field.

Nice To Haves

Experience with additional AWS services, including API Gateway, ElasticSearch, and SQS.
Experience with infrastructure-as-code tools such as Terraform or CloudFormation.
Experience with DevOps practices and CI/CD pipelines.
Experience implementing end-to-end streaming solutions using Amazon Kinesis, SQS, and Kafka.
AWS Solutions Architect or AWS Developer certification.
Understanding of Lakehouse/data cloud architecture.
Knowledge of data governance and compliance standards.

Responsibilities

Design, build, and maintain efficient, reusable, and reliable architecture and code for data pipelines and data applications on AWS.
Build data ingestion pipelines from on-premises environments to AWS and within AWS using services such as Glue, Redshift, S3, Lambda, EMR/Spark, Kinesis, and SQS.
Develop and manage ETL/ELT processes that collect, process, and store data from multiple sources while ensuring data quality, integrity, and security.
Architect and implement end-to-end AWS data solutions for ingestion, storage, integration, processing, and access, particularly data lakes and data warehouses.
Participate in architecture and system design discussions for high-scale data engineering projects.
Perform hands-on development and unit testing, and participate in code reviews to ensure adherence to best practices.
Implement serverless applications using AWS Lambda, API Gateway, Step Functions, and other AWS technologies.
Migrate data from relational databases, file systems, and APIs to AWS-based data lakes, RDS, Aurora, and Redshift.
Implement high-velocity streaming solutions using Amazon Kinesis and SQS, with Kafka identified as preferred.
Architect and implement CI/CD strategies for enterprise data platforms.
Collaborate with product, operations, QA, and cross-functional teams throughout the software development cycle.
Track new technology developments, create proofs of concept for tools and technologies, and onboard them for real-world use cases.
Identify and resolve performance issues and optimize solutions for cost, reliability, and scalability.
Oversee organizational data engineering solutions, lead high-impact initiatives, ensure alignment with organizational goals, and provide executive guidance.