Data Pipeline Architect

Stefanini Group•Dearborn, MI

1d•Onsite

About The Position

Stefanini Group is hiring a Data Pipeline Architect in Dearborn, MI. This role is pivotal in the development and maintenance of our Enterprise Data Platform, focusing on designing, building, and optimizing scalable data pipelines within a Google Cloud Platform (GCP) environment. The architect will work with GCP Native technologies like BigQuery, Dataflow, and Pub/Sub, ensuring data governance, security, and optimal performance.

Requirements

Bachelor's degree in Computer Science, Information Technology, Information Systems, Data Analytics, or a related field (or equivalent combination of education and experience).
5-7 years of experience in Data Engineering or Software Engineering, with at least 2 years of hands-on experience building and deploying cloud-based data platforms (GCP preferred).
Strong proficiency in SQL, Java, and Python, with practical experience in designing and deploying cloud-based data pipelines using GCP services like BigQuery, Dataflow, and DataProc.
Solid understanding of Service-Oriented Architecture (SOA) and microservices, and their application within a cloud data platform.
Experience with relational databases (e.g., PostgreSQL, MySQL), NoSQL databases, and columnar databases (e.g., BigQuery).
Knowledge of data governance frameworks, data encryption, and data masking techniques in cloud environments.
Familiarity with CI/CD pipelines, Infrastructure as Code (IaC) tools like Terraform and Tekton, and other automation frameworks.
Excellent analytical and problem-solving skills, with the ability to troubleshoot complex data platform and microservices issues.
Experience in monitoring and optimizing cost and compute resources for processes in GCP technologies (e.g., BigQuery, Dataflow, Cloud Run, DataProc).

Nice To Haves

Data/Analytics dashboards

Responsibilities

Spearhead the design, development, and maintenance of scalable data ingestion and curation pipelines from diverse sources.
Ensure data is standardized, high-quality, and optimized for analytical use.
Leverage cutting-edge tools and technologies, including Python, SQL, and DBT/Dataform, to build robust and efficient data pipelines.
Utilize full-stack skills to contribute to seamless end-to-end development, ensuring smooth and reliable data flow from source to insight.
Leverage deep expertise in GCP services (BigQuery, Dataflow, Pub/Sub, Cloud Functions, etc.) to build and manage data platforms that meet business needs.
Implement and manage robust data governance policies, access controls, and security best practices, utilizing GCP's native security features.
Employ Astronomer and Terraform for efficient data workflow management and cloud infrastructure provisioning, championing best practices in Infrastructure as Code (IaC).
Continuously monitor and improve the performance, scalability, and efficiency of data pipelines and storage solutions.
Collaborate effectively with data architects, application architects, service owners, and cross-functional teams to define and promote best practices, design patterns, and frameworks for cloud data engineering.
Proactively automate data platform processes to enhance reliability, improve data quality, minimize manual intervention, and drive operational efficiency.