[C]-Developer - Application Development Level 5

V R Della Infotech • New York City, NY

7d • Hybrid

About The Position

The Scala Sparks Developer will develop, test, and maintain scalable data-processing pipelines using Scala and Apache Spark. The role includes implementing data transformation and ETL workflows for structured and unstructured data and using Python for data processing, scripting, and integration with Spark workflows. The developer will optimize Spark application performance through tuning, partitioning, and caching, participate in code reviews and design discussions, follow best practices, and document workflows, architecture, and solutions for internal knowledge sharing. The position is listed in NYC, NY, with a minimum of three days onsite per week, for a six-month duration.

Requirements

• 8+ years of experience.
• Strong experience with Scala and Apache Spark, including Spark Core, Spark SQL, DataFrames, and RDDs.
• Proficiency in Python for data manipulation, scripting, and automation.
• Proven experience with distributed computing concepts and large-scale data processing.
• Knowledge of ETL development, data warehousing, and data modeling techniques.
• Excellent communication and collaboration skills.

Nice To Haves

• Familiarity with Big Data ecosystems such as Hadoop, Hive, Kafka, or HBase.
• Experience with cloud platforms such as AWS, GCP, or Azure, particularly Spark services.
• Familiarity with streaming data processing using Spark Streaming or Structured Streaming.
• Knowledge of data governance, security, and compliance practices.
• Basic understanding of machine learning workflows.

Responsibilities

• Develop, test, and maintain scalable data-processing pipelines using Scala and Apache Spark.
• Implement data transformation and ETL workflows for structured and unstructured data.
• Use Python for data processing, scripting, and integration of Spark workflows.
• Optimize Spark application performance through tuning, partitioning, and caching strategies.
• Participate in code reviews and design discussions and ensure adherence to best practices.
• Document workflows, architecture, and solutions for internal knowledge sharing.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed