Senior GCP Data Engineer

Comspark group•Sunnyvale, CA
•Onsite

About The Position

The Senior GCP Data Engineer designs and develops big data applications using current open-source technologies. The role involves developing logical and physical data models for big data platforms, automating workflows with Apache Airflow, and creating data pipelines with Apache Hive, Apache Spark, and Apache Kafka. The engineer maintains and enhances existing systems and participates in rotational on-call support. The position also involves learning the business domain and technology infrastructure, sharing knowledge with the team, mentoring junior engineers, leading daily standups and design reviews, grooming and prioritizing the backlog using JIRA, and serving as the point of contact for an assigned business domain. The posting describes a Sunnyvale, California position requiring five days onsite and mentions working in an offshore model with managed outcomes as desired.

Requirements

  • At least 4 years of recent GCP experience.
  • Experience building data pipelines in GCP.
  • Experience with GCP Dataproc, GCS, and BigQuery.
  • At least 12 years of hands-on experience developing data warehouse solutions and data products.
  • At least 6 years of hands-on experience developing a distributed data processing platform using Hadoop, Hive, or Spark, and Airflow or another workflow orchestration solution.
  • At least 5 years of hands-on experience modeling and designing schemas for data lakes or RDBMS platforms.
  • Experience with programming languages such as Python, Java, or Scala.
  • Experience with scripting languages such as Perl or Shell.
  • Experience processing and managing large datasets at multi-terabyte or petabyte scale.
  • Exposure to test-driven development and automated testing frameworks.
  • Background in Scrum or Agile development methodologies.
  • Ability to deliver on multiple competing priorities with little supervision.
  • Excellent verbal and written communication skills.
  • Bachelor's degree in computer science or equivalent experience.

Nice To Haves

  • Experience with Gitflow.
  • Experience with Atlassian products such as Bitbucket, JIRA, and Confluence.
  • Experience with continuous integration tools such as Bamboo, Jenkins, or TFS.
  • Experience working in an offshore model and with managed outcomes.

Responsibilities

  • Design and develop big data applications using open-source technologies.
  • Develop logical and physical data models for big data platforms.
  • Automate workflows using Apache Airflow.
  • Create data pipelines using Apache Hive, Apache Spark, and Apache Kafka.
  • Provide ongoing maintenance and enhancements to existing systems.
  • Participate in rotational on-call support.
  • Learn the business domain and technology infrastructure and share knowledge with team members.
  • Mentor junior engineers.
  • Lead daily standups and design reviews.
  • Groom and prioritize the backlog using JIRA.
  • Act as the point of contact for an assigned business domain.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service