AWS Data Engineer 3851

Intelliswift Software • Santa Clara, CA

96d

About The Position

The AWS Data Engineer will be responsible for the design, development, and management of data integration processes. This includes integrating data from diverse sources, transforming it to meet business requirements, and loading it into target systems such as data warehouses or data lakes. The role involves developing and maintaining data integration solutions, ensuring data quality and integrity, optimizing data integration processes, supporting business intelligence and analytics, and maintaining documentation and compliance.

Requirements

• Bachelor's degree in computer science, information technology, or a related field; a master's degree is advantageous.
• 4-6+ years of experience in data engineering, database design, and ETL processes.
• Experience with Iceberg.
• 5+ years in programming languages such as PySpark, Python, and SQL.
• 5+ years of experience with AWS tools and technologies (S3, EMR, Glue, Athena, RedShift, Postgres, RDS, Lambda, PySpark).
• 3+ years of experience working with databases, data marts, and data warehouses.
• Experience in ETL development, system integration, and CI/CD implementation.
• Solid understanding of data security, privacy, and compliance.

Nice To Haves

• Experience in complex database objects to move changed data across multiple environments.
• Participation in agile development processes including sprint planning, stand-ups, and retrospectives.
• Ability to provide technical guidance and mentorship to junior developers.

Responsibilities

• Design and implement data integration workflows using AWS Glue, EMR, Lambda, Redshift, PySpark, Spark, and Python for processing large datasets.
• Ensure data is extracted, transformed, and loaded into target systems.
• Build ETL pipelines using Iceberg.
• Validate and cleanse data to ensure quality and integrity.
• Implement monitoring, validation, and error handling mechanisms within data pipelines.
• Enhance performance and optimize data workflows to meet SLAs and scalability on AWS cloud infrastructure.
• Resolve performance bottlenecks and optimize data processing to enhance Redshift's performance.
• Translate business requirements to technical specifications and coded data pipelines.
• Document all data integration processes, workflows, and technical & system specifications.
• Ensure compliance with data governance policies, industry standards, and regulatory requirements.

What This Job Offers

• Industry: Professional, Scientific, and Technical Services
• Education Level: Bachelor's degree