PySpark Developer

Wipro•Minneapolis, MN

9d•Hybrid

About The Position

Wipro Technologies-USA invites applications for the position of Solution Architect L1, specializing as a PySpark Developer within our esteemed Technology Services sector. This pivotal role is part of the TS - DATA ANALYTICS & AI group, focusing on Data Transformation initiatives that drive our clients' digital evolution. As a PySpark Developer, you will leverage your extensive expertise in Spark Open Source technology to design, architect, and implement scalable big data solutions that meet complex business objectives. You will collaborate closely with cross-functional teams, including data engineers, analysts, and business stakeholders, to translate data requirements into robust architectures, ensuring high performance and reliability. Key competencies essential for success include being respectful in all interactions, maintaining clear and consistent communication, demonstrating stewardship over data assets and solutions, responding promptly to evolving project needs, and building trust within the team and with client partners. Your role will involve leading technical discussions, crafting best practice guidelines, and ensuring compliance with industry standards. Candidates considered for this role should possess between 8 to 10 years of professional experience in data analytics, specifically working with Spark Open Source technologies. A strong understanding of distributed computing, data processing frameworks, and cloud platforms will be advantageous. The position demands a strategic thinker with a hands-on approach, able to balance architectural oversight with practical coding when required. The role is located in Irving, TX, with a requirement of 3 days onsite per week. We are looking for an experienced PySpark Developer with over 7 years of hands-on experience in designing, developing, and optimizing large-scale data processing applications. The ideal candidate should have strong expertise in PySpark, Azure cloud services, Docker, and Azure Kubernetes Service (AKS), with the ability to build scalable, secure, and high-performing data solutions.

Requirements

Strong hands-on experience in PySpark development for distributed data processing.
Experience with Azure Kubernetes Service (AKS) for deploying and managing containerized workloads.
Proficiency in Docker for containerization, image management, and deployment workflows.
Solid working knowledge of Azure cloud services and cloud-native application development.
Strong understanding of data engineering concepts, ETL/ELT processes, and performance optimization.
Ability to troubleshoot complex technical issues and deliver timely resolutions.
7 years of overall experience in data engineering, big data development, or related technology roles.
Strong practical experience in building and supporting production-grade PySpark-based data pipelines.
Experience working in cloud-based environments, preferably Microsoft Azure.
Good understanding of CI/CD practices, DevOps workflows, and container-based deployments.
Strong analytical, problem-solving, and communication skills.
Ability to work independently as well as collaboratively in a fast-paced delivery environment.
Spark Open Source

Nice To Haves

Working knowledge of Python for scripting, automation, and data processing support.
Experience with SSIS for ETL workflow development and migration activities.
Exposure to Informatica for data integration and enterprise data management.
Experience in identifying, analyzing, and fixing security vulnerabilities across applications, containers, and cloud deployments.

Responsibilities

Design, develop, and maintain scalable data pipelines using PySpark for large-volume data processing.
Work with Azure cloud services to build reliable and efficient data engineering solutions.
Deploy, manage, and monitor containerized applications using Docker and Azure Kubernetes Service (AKS).
Collaborate with data architects, analysts, and business stakeholders to understand requirements and deliver high-quality data solutions.
Optimize PySpark jobs for performance, scalability, reliability, and cost efficiency.
Implement best practices for data ingestion, transformation, validation, and processing across cloud environments.
Identify and fix application, container, and platform vulnerabilities to ensure secure deployments.
Support troubleshooting, production issue resolution, and performance tuning of data pipelines and cloud-based applications.
Participate in code reviews, technical discussions, deployment planning, and documentation activities.