GCP Data Engineer (AI Experience)

VSG Business Solutions•Hartford, CT

About The Position

The GCP Data Engineer will design, build, and maintain data pipelines and workflows using Google Cloud Platform services. The role includes developing scalable Python applications and complex BigQuery queries, orchestrating workflows with Cloud Composer (Apache Airflow), and building real-time data processing systems with Apache Kafka. The engineer will also work with MongoDB and other NoSQL data, support data infrastructure with shell scripting and Linux administration, and ensure data quality through profiling and validation. The position requires experience applying AI tools such as GPT and Gemini to business use cases, with that experience clearly demonstrated on the resume. The engineer will troubleshoot data issues, develop API integration scripts, and create data flow diagrams to explain architectures and pipeline logic. The role involves independent and collaborative work in a fast-paced environment, including coordination with offshore teams in an onshore/offshore support model and participation in Agile practices.

Requirements

At least 5 years of proven hands-on experience with GCP data services.
At least 3 years of proven hands-on experience with the listed data engineering technologies and responsibilities, including GCP services, Cloud Composer, Python, BigQuery SQL, Kafka, MongoDB, shell scripting, Linux administration, data quality, APIs, and data flow diagrams.
Experience using AI tools such as GPT and Gemini, with business use cases clearly highlighted on the resume; candidates must be able to demonstrate their use.
Strong understanding of distributed systems and data engineering best practices.
Excellent communication, documentation, and collaboration skills.
Ability to work independently and collaboratively in a fast-paced environment.
Experience with Agile methodology.
Experience working in an onshore/offshore support model and collaborating with offshore teams.
ROPES assessment required.

Nice To Haves

Google Cloud Professional Data Engineer certification.
Experience with SAFe methodology.
Experience working with GitHub, RTC, and automation tools.

Responsibilities

Design and implement data pipelines using GCP services, including BigQuery, Cloud Storage, and Pub/Sub.
Develop and manage workflow scheduling and orchestration using Cloud Composer (Apache Airflow).
Write clean, efficient, scalable Python code using advanced programming techniques.
Create complex BigQuery SQL queries using window functions and CTEs, and apply performance-tuning strategies.
Build and maintain real-time data processing systems using Apache Kafka.
Model and manage MongoDB and other NoSQL databases, with attention to scalable schema design.
Use shell scripting and perform Linux system administration to support data infrastructure.
Conduct data profiling and implement validation techniques to maintain data quality and integrity.
Develop and maintain API integration scripts for service automation and data exchange.
Troubleshoot and resolve data-related issues.
Create and maintain data flow diagrams to communicate architecture and pipeline logic to stakeholders.
Demonstrate practical use of AI tools such as GPT and Gemini for business use cases.
Collaborate with offshore teams within an onshore/offshore support model.