Big Data Engineer

Codetech • Malvern, PA

10h

About The Position

Analyze complex business problems, identify required information, document requirements, and develop technical solutions. Troubleshoot and optimize new and existing ETL packages. Build Spark data-processing applications using Python and PySpark, develop SQL, and work with AWS data and cloud services. The role also involves creating AWS CloudFormation templates and server-level architecture frameworks in AWS.

Requirements

• Strong analytical and problem-solving skills and the tenacity to overcome technical challenges.
• Experience with AWS EMR, AWS Glue, AWS Athena, AWS CloudFormation, AWS IAM, AWS S3, and AWS SNS.
• Experience with SQL development; SQL is listed as a basic qualification.
• Experience with CI/CD pipeline tools such as Bamboo, Bitbucket, or GitHub.
• Experience with Test Driven Development.
• Experience building Spark data-processing applications using Python and PySpark.

Nice To Haves

• Experience with Unix scripting and Linux configuration.
• Experience with AWS Step Functions and AWS Lambda.
• Experience with Tableau and Hive.

Responsibilities

• Analyze complex business problems, track down required information, document requirements, and develop technical solutions.
• Troubleshoot and optimize new and existing ETL packages.
• Build Spark data-processing applications using Python and PySpark.
• Develop SQL.
• Build AWS CloudFormation templates and server-level architecture frameworks in AWS.

What This Job Offers

• Job Type: Full-time
• Education Level: No Education Listed