Senior Big Data Engineer – Scala/Spark
About The Position
We are seeking a Senior Big Data Engineer with strong expertise in Scala and Spark to join our team. The role involves designing, developing, and maintaining large-scale, distributed data processing pipelines. You will work in an Agile environment, collaborating with cross-functional teams to translate project requirements into technical solutions and troubleshoot data issues. The ideal candidate will have a minimum of 8 years of experience in Big Data technologies, a strong understanding of data structures, database concepts, and ETL techniques. Experience in a global delivery environment and a commitment to staying updated with new technologies are also important.
Requirements
- At least 8 years of experience and strong knowledge in Scala programming language.
- Able to write clean, maintainable and efficient Scala code following best practices.
- Good knowledge on the fundamental Data Structures and their usage.
- At least 8 years of experience in designing and developing large scale, distributed data processing pipelines using Apache Spark and related technologies.
- Expertise in Spark Core, Spark SQL and Spark Streaming.
- Experience with Python, Hadoop, HDFS, Hive and other BigData technologies.
- Familiarity with Data warehousing and ETL concepts and techniques.
- Expertise in Database concepts and SQL/NoSQL operations.
- At least 8 years of experience in Project development life cycle activities and maintenance/support projects.
- Work in an Agile environment and participation in scrum daily standups, sprint planning reviews and retrospectives.
- Understand project requirements and translate them into technical solutions which meets the project quality standards.
- Ability to work in team in diverse/multiple stakeholder environment and collaborate with upstream/downstream functional teams to identify, troubleshoot and resolve data issues.
- Strong problem solving and Good Analytical skills.
- Excellent verbal and written communication skills.
- Experience and desire to work in a Global delivery environment.
- Stay up to date with new technologies and industry trends in Development.
Nice To Haves
- Spark Streaming
- Hadoop
- Hive
- SQL
- Sqoop
- Impala
- UNIX shell scripting will be an added advantage in scheduling/running application jobs.
Responsibilities
- Coding
- Requirement gathering
- Team lead
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
