Senior GCP Data Engineer

Persistent Systems•Charlotte, NC

11h•Hybrid

About The Position

Persistent is seeking a Senior GCP Data Engineer to design, build, deploy, and optimize scalable data pipelines and enterprise data integration solutions. The role focuses on Google Cloud Platform services, including BigQuery, Dataflow, Cloud Composer, and Cloud Storage, with mandatory hands-on experience in Starburst and/or Dremio for data virtualization, query federation, and analytics across distributed data sources. The engineer will also work with MongoDB and Python, and help ensure data platform performance, reliability, quality, governance, and security. The position additionally involves developing AI agents and survey-programming workflows using Python, LangGraph, Microsoft Agent Framework, and Azure OpenAI. The engineer will collaborate with data scientists, software engineers, architects, and business stakeholders, while supporting production environments, CI/CD, operational procedures, and technical documentation.

Requirements

7–12 years of experience in Data Engineering, Cloud Data Platforms, or Big Data environments.
Strong hands-on experience with Google Cloud Platform (GCP).
Expertise in BigQuery, Cloud Dataflow, Cloud Composer (Apache Airflow), and Cloud Storage.
Mandatory hands-on experience with Starburst and/or Dremio.
Strong experience implementing ETL/ELT pipelines and enterprise data integration solutions.
Advanced Python programming and scripting expertise.
Strong SQL development skills and query optimization experience.
Experience working with MongoDB and NoSQL database technologies.
Solid understanding of data warehousing, dimensional modeling, and analytical data platforms.
Hands-on experience monitoring and supporting production-grade data environments.
Experience with Cloud Logging, Cloud Monitoring, and incident troubleshooting.
Knowledge of CI/CD implementation and DevOps practices for data engineering deployments.
Strong analytical, troubleshooting, and problem-solving skills.
Experience working in Agile and cross-functional teams.
Excellent communication and stakeholder management abilities.

Responsibilities

Design, develop, and deploy AI agents using Python, LangGraph, Microsoft Agent Framework, and Azure OpenAI.
Build and optimize Survey Programming Agents for questionnaire parsing, routing logic, JSON generation, dimensions setup, STG generation, translation workflows, DMS script creation, link testing, survey quality control, and change impact analysis.
Design, develop, and deploy scalable ETL/ELT data pipelines using Python and GCP-native services.
Build and maintain enterprise-grade data integration solutions across multiple data sources and platforms.
Integrate MongoDB data into Google Cloud Storage and BigQuery environments.
Design and optimize data ingestion, transformation, and analytical processing workflows.
Develop and support data virtualization and federated query solutions using Starburst and/or Dremio.
Optimize BigQuery workloads through partitioning, clustering, query tuning, and performance engineering.
Monitor and troubleshoot production data pipelines and resolve data quality issues.
Debug data processing failures, perform root cause analysis, and resolve issues.
Optimize Dataflow jobs and MongoDB indexing strategies to improve performance and reduce operational costs.
Implement monitoring, alerting, and operational support processes for data platforms.
Collaborate with Data Scientists, Software Engineers, Architects, and business stakeholders to deliver reliable data solutions.
Develop and maintain technical documentation, operational runbooks, and deployment procedures.
Implement CI/CD processes for data engineering solutions and cloud deployments.
Ensure adherence to data governance, security, compliance, and operational best practices.

Benefits

Competitive salary and benefits package.
Culture focused on talent development, with quarterly growth opportunities.
Company-sponsored higher education and certifications.
Opportunity to work with cutting-edge technologies.
Employee engagement initiatives, including project parties.
Flexible work hours.
Long Service awards.
Annual health check-ups.
Group term life, personal accident, and Mediclaim hospitalization insurance coverage for the employee, spouse, two children, and parents.
Values-driven, people-centric, and inclusive work environment.
Hybrid work and flexible hours.
Accessibility-friendly office with ergonomic setups and assistive technologies.