GCP Data Engineer

Recruiting Source International•Downey, CA

About The Position

We are seeking a skilled GCP Data Engineer to design, build, and maintain scalable data solutions on Google Cloud Platform. This role involves architecting data ingestion, transformation, and visualization workflows, ensuring data quality and security, and collaborating with cross-functional teams to deliver actionable insights. You will leverage GCP services such as BigQuery, Dataflow, Cloud Storage, Cloud SQL, and Cloud Functions to meet complex data processing needs.

Requirements

Proficiency in writing dbt models (sources, models, tests, snapshots), managing dbt projects, and utilizing dbt's features (version control, documentation, testing).
Experience with data ingestion and transformation techniques.
Understanding and implementing data extraction, transformation, and loading processes.
Implementing data quality checks and validations within dbt models (e.g., data type checks, uniqueness constraints, null checks).
Utilizing dbt's testing framework or other testing tools to ensure data accuracy and integrity.
Integrate data from diverse sources (on-premise, cloud-based applications, APIs) into BigQuery using data ingestion tools and techniques.
Develop data integration workflows to ensure data consistency and quality across different systems.
Leverage other GCP services like Cloud Dataflow for real-time data processing and stream analytics.
Work closely with data analysts, business stakeholders, and software engineers to understand data requirements and translate them into actionable insights.
Document technical design decisions and provide clear explanations of complex data concepts.
Proactively identify potential issues and propose solutions to ensure data quality and system reliability.
Deep understanding of data warehousing principles, data modeling, and dimensional design.
Expertise in SQL and proficiency in other programming languages like Python for data manipulation, automation, and interacting with APIs.
Familiarity with data visualization tools to present insights effectively.
Knowledge of cloud computing concepts and best practices on Google Cloud Platform.
Proven experience in designing and implementing data solutions using Google BigQuery and other GCP services.
Strong SQL skills with expertise in complex query optimization.
Excellent problem-solving and analytical abilities.
Ability to work independently and as part of a team.

Nice To Haves

Google Cloud Platform certifications (e.g., Certified Professional Data Engineer, Certified Professional Cloud Architect) are highly desirable.

Responsibilities

Evaluate data ingestion methods, data transformation pipelines, and data visualization techniques to optimize data analysis workflows.
Create data models and schema designs that facilitate efficient querying and data retrieval.
Ensure data security and compliance with relevant regulations by implementing appropriate access controls and encryption strategies.
Assist in the review and enhancements of the data architectures leveraging GCP services like BigQuery, Dataflow, Cloud Storage, Cloud SQL, and Cloud Functions to meet complex data processing needs.
Build and maintain robust data pipelines using tools like Apache Airflow, Prefect, or similar.
Implement data extraction, transformation, and loading processes.
Implement data quality checks and validations within dbt models (e.g., data type checks, uniqueness constraints, null checks).
Utilize dbt's testing framework or other testing tools to ensure data accuracy and integrity.
Integrate data from diverse sources (on-premise, cloud-based applications, APIs) into BigQuery using data ingestion tools and techniques.
Develop data integration workflows to ensure data consistency and quality across different systems.
Leverage other GCP services like Cloud Dataflow for real-time data processing and stream analytics.
Work closely with data analysts, business stakeholders, and software engineers to understand data requirements and translate them into actionable insights.
Document technical design decisions and provide clear explanations of complex data concepts.
Proactively identify potential issues and propose solutions to ensure data quality and system reliability.