GCP Data Engineer (AI Experience)
About The Position
The GCP Data Engineer will design, build, and maintain data pipelines and data infrastructure on Google Cloud Platform. The role uses GCP services including BigQuery, Cloud Storage, Pub/Sub, and Cloud Composer 3, along with Python, SQL, Apache Kafka, and MongoDB. The engineer will support data quality, integrations, workflow orchestration, troubleshooting, and clear communication of data architecture. The position requires local candidates able to work onsite in Hartford, Connecticut three days per week. A Ropes assessment is required. Applicants must have AI experience with GPT and Gemini and be able to demonstrate business use cases, which must be clearly highlighted on the resume.
Requirements
- Minimum 7 years of proven hands-on experience with GCP data services.
- Minimum 3 years of proven hands-on experience across the listed data engineering technologies and responsibilities, including GCP services, Cloud Composer, Python, BigQuery SQL, Apache Kafka, MongoDB, shell scripting, Linux, data quality, and API integrations.
- Strong understanding of distributed systems and data engineering best practices.
- Excellent communication and documentation skills.
- Ability to work independently and collaboratively in a fast-paced environment.
- Experience with Agile methodology.
- Good communication and collaborative skills.
- Experience working in an onshore/offshore support model with offshore teams.
- Must have experience with AI tools GPT and Gemini, be able to demonstrate their use in business use cases, and clearly highlight this experience on the resume.
- Must be local to Hartford, Connecticut and able to work onsite three days per week.
- Must complete a Ropes assessment.
Nice To Haves
- Google Cloud Professional Data Engineer certification.
- Experience with SAFe methodology.
- Experience working with GitHub, RTC, and automation tools.
Responsibilities
- Design and implement robust data pipelines using GCP services such as BigQuery, Cloud Storage, and Pub/Sub.
- Develop and manage workflows using Cloud Composer (Apache Airflow), including Cloud Composer 3, for scheduling and orchestration.
- Write clean, efficient, and scalable Python code using advanced programming techniques.
- Write complex BigQuery SQL queries, including window functions and CTEs, and apply performance-tuning strategies.
- Build and maintain real-time data processing systems using Apache Kafka.
- Model and manage MongoDB databases, focusing on scalable schema design.
- Use shell scripting and perform Linux system administration tasks to support data infrastructure.
- Conduct data profiling and implement validation techniques to ensure data quality and integrity.
- Develop and maintain API integration scripts for service automation and data exchange.
- Troubleshoot and resolve data-related issues.
- Create and maintain data flow diagrams to communicate architecture and pipeline logic to stakeholders.
- Collaborate with offshore teams in an onshore/offshore support model.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Education Level
No Education Listed
