Hadoop Developer

Agama Solutions•Addison, TX

4d•Hybrid

About The Position

This is a 12-month contract position for a Hadoop Developer with Python skills, located in Addison, TX. The role involves working in a hybrid environment and requires strong expertise in Hadoop, Python/Scala, and SparkSQL. The developer will be responsible for designing and building data pipelines, performing performance tuning, and working with various big data technologies.

Requirements

Hadoop
Python/Scala
SparkSQL
Strong SQL Skills (MySQL, HIVE, Impala, SPARK SQL)
Data ingestion experience (message queue, file share, REST API, relational database)
Experience with data formats (json, csv, xml)
Experience working with SPARK Structured streaming
Experience working with Hadoop/Big Data and Distributed Systems
Working experience with Spark, Sqoop, Kafka, MapReduce, NoSQL Database like HBase, SOLR, CDP or HDP, Cloudera or Hortonworks, Elastic Search, Kibana
Hands on programming experience in at least one of Scala, Python, PHP, or Shell Scripting
Performance tuning experience with spark /MapReduce or SQL jobs
Experience and proficiency with Linux operating system
Experience in end-to-end design and build process of Near-Real Time and Batch Data Pipelines
Experience working in Agile development process
Deep understanding of various phases of the Software Development Life Cycle
Experience using Source Code and Version Control systems like SVN, Git, Bit Bucket
Experience working with Jenkins and Jar management
Self-starter who works with minimal supervision
Ability to work in a team of diverse skill sets
Ability to comprehend customer requests and provide the correct solution
Strong analytical mind
Desire to resolve issues and dive into potential issues
Ability to adapt and continue to learn new technologies

Responsibilities

Design and build Near-Real Time and Batch Data Pipelines.
Perform performance tuning on Spark/MapReduce or SQL jobs.
Ingest data from various sources including message queues, file shares, REST APIs, and relational databases.
Work with data formats like JSON, CSV, and XML.
Utilize Spark Structured Streaming.
Work with Hadoop/Big Data and Distributed Systems.
Use Spark, Sqoop, Kafka, MapReduce, and NoSQL databases like HBase, SOLR.
Develop solutions using Scala, Python, PHP, or Shell Scripting.
Work with Linux operating system.
Understand and apply various phases of the Software Development Life Cycle.
Utilize Source Code and Version Control systems like SVN, Git, Bit Bucket.
Work with Jenkins and Jar management.
Comprehend customer requests and provide solutions.
Analyze and solve complicated problems.
Resolve issues and investigate potential problems.
Adapt and learn new technologies.