Senior Python Platform Engineer – PySpark / FastAPI / GCP

Infosys Pontoon • Charlotte, NC

2d • Onsite

About The Position

We are seeking a Python-based platform engineer to design and build a containerized API layer that abstracts and governs interactions with Apache Spark through a well-defined API contract. This role focuses on building platform capabilities, not simply consuming existing data tools—enabling consistent, secure, and scalable access to Spark-based data pipelines. The ideal candidate has strong experience developing production-grade APIs in Python that interface with data frameworks or pipeline orchestration systems, packaging services using containers (Docker/Kubernetes), and operating them as reusable platform services. A proven background in CI/CD automation using GitHub Actions is required, along with solid software engineering practices around testing, versioning, and deployment. Should have good hands-on experience in GCP skills. Should be able to interact, coordinate with the client. Should be able to manage offshore resources also. Should be able to work with other onsite team members and client.

Requirements

• At least 10 years of experience in IT industry.
• Python
• PySpark
• API development
• Microservices
• CI/CD
• GCP
• ETL
• Experience building control planes
• Design patterns: Modular containerized services, Asynchronous processing, Job-based work queue execution, Immutability and idempotency, Metadata management, Policy enforcement, Semantic modeling, Observability and telemetry, Side effect capability injection
• GCP Testing: Functional testing, Behavioral testing
• FastAPI
• ARQ
• Kubernetes
• Pytest

Nice To Haves

• ETL

Responsibilities

• Design and build a containerized API layer that abstracts and governs interactions with Apache Spark through a well-defined API contract.
• Build platform capabilities, not simply consuming existing data tools.
• Enable consistent, secure, and scalable access to Spark-based data pipelines.
• Develop production-grade APIs in Python that interface with data frameworks or pipeline orchestration systems.
• Package services using containers (Docker/Kubernetes) and operate them as reusable platform services.
• Implement CI/CD automation using GitHub Actions.
• Apply solid software engineering practices around testing, versioning, and deployment.
• Interact and coordinate with the client.
• Manage offshore resources.
• Work with other onsite team members and client.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed