Data Engineer (AWS EMR)

TalentOla • Chicago, IL

5d • Onsite

About The Position

As a Data Engineer with deep expertise in Python, AWS, big data ecosystems, and SQL/NoSQL technologies, driving scalable, real-time data solutions with CI/CD and stream-processing frameworks.

Requirements

• AWS
• EMR
• Python
• PySpark
• SQL (complex queries, relational databases, preferably PostgreSQL, and NoSQL databases - Redis and Elasticsearch)
• AWS Lambda
• AWS Glue
• AWS Athena
• AWS Kinesis
• AWS IAM
• AWS EMR/PySpark
• Docker
• Git
• Terraform
• Agile methodologies
• Stream-processing systems (Storm, Spark-Streaming)
• Workflow management tools (Airflow)

Nice To Haves

• Spark
• Kafka/Kinesis
• Knowledge graph technologies (Graph DB, OWL, SPARQL)

Responsibilities

• Proficient developer in multiple languages, Python is a must, with the ability to quickly learn new ones.
• Expertise in SQL (complex queries, relational databases, preferably PostgreSQL, and NoSQL databases - Redis and Elasticsearch).
• Extensive big data experience, including EMR, Spark, Kafka/ Kinesis, and optimising data pipelines, architectures, and datasets.
• AWS expert with hands-on experience in Lambda, Glue, Athena, Kinesis, IAM, EMR/PySpark, Docker.
• Proficient in CI/CD development using Git, Terraform, and agile methodologies.
• Comfortable with stream-processing systems (Storm, Spark-Streaming) and workflow management tools (Airflow).

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed