Senior Spark QA Engineer

InfraCloud Technologies

7d • Remote

About The Position

We are looking for a Senior Spark QA Engineer with strong expertise in functional and performance testing of Apache Spark applications and distributed data platforms.

Requirements

• 5+ years of QA/testing experience with strong hands-on expertise in Apache Spark.
• Experience in functional and performance testing of distributed systems.
• Strong understanding of Spark architecture and optimization techniques.
• Experience with Kubernetes and cloud-based Spark environments.
• Proficiency in Python or Java.
• Experience with automation frameworks and CI/CD pipelines.

Responsibilities

• Perform manual and automated testing of Spark jobs, Spark SQL queries, and ETL pipelines.
• Execute functional, scalability, and performance testing for Spark workloads.
• Set up and manage Spark clusters on Standalone, YARN, Kubernetes, Mesos, Databricks, EMR, and Dataproc.
• Conduct benchmarking, validation, and performance analysis of Spark applications.
• Identify bottlenecks and troubleshoot distributed system issues.
• Lead QA initiatives and mentor team members.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed