AWS Data Migration Engineer

Algonomy • Philadelphia, PA

34d • Onsite

About The Position

This position is responsible for the support, migration, automation, and monitoring of existing data pipelines. Based on new requirements, the role involves designing, developing, testing, and deploying Big Data Analytics solutions on cloud (AWS) and Databricks. The engineer will collaborate with key stakeholders to translate business requirements into technical requirements and implement solutions. A significant part of the role involves working with large, complex customer TV and Digital Ad viewing data to perform pre-campaign planning and post-campaign performance measurements using Big data platforms like Databricks, PySpark, Glue, and EMR. The engineer will also be responsible for supporting, migrating, automating, and monitoring data pipelines to ingest data, integrate data from multiple sources (On-Premise & Cloud), and create aggregated datasets for reporting needs, along with appropriate notifications.

Requirements

• 8+ years of overall software development experience.
• Experience with Big Data Technologies and Hadoop Ecosystem.
• Programming experience in Scala, PySpark, and Python.
• Experience in requirement gathering, design, support, migration, and automation.
• Experience in monitoring data pipelines with PySpark and AWS services.

Responsibilities

• Support, migration, automation, and monitoring of existing data pipelines.
• Design, develop, test, and deploy Big Data Analytics solutions on cloud (AWS) and Databricks based on new requirements.
• Collaborate with key stakeholders to translate business requirements into technical requirements and implement solutions.
• Analyze large, complex customer TV and Digital Ad viewing data for pre-campaign planning and post-campaign performance measurement using Big data platforms (Databricks, PySpark, Glue, EMR).
• Support, migrate, automate, and monitor data pipelines for data ingestion, integration from multiple sources (On-Premise & Cloud), and creation of aggregated datasets for reporting.
• Implement appropriate notifications for reporting needs.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed