Lead Data Engineer - Technology Data Product
About The Position
KeyBank is seeking a Lead Data Engineer to support the design, implementation, maintenance, and lifecycle of enterprise data products, beginning with a DevOps Data Product that transforms technology, operational, and business data into trusted, governed, and reusable insights. In this hands-on technical leadership role, the Lead Data Engineer will partner with Data Product Owners, data modelers, data architects, platform engineers, technology teams, and business stakeholders to develop scalable data pipelines, reusable data models, cloud-native services, and deployment automation on Google Cloud Platform. The role focuses on solving complex data and platform problems, improving data quality and reliability, reducing friction in data delivery and the pipeline lifecycle, and building standardized solutions that encourage broad adoption. The Lead Data Engineer will influence data engineering standards, strengthen platform capabilities, and help shape a delivery ecosystem built for speed, quality, governance, and resilience.
Requirements
- Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related technical discipline, or equivalent practical experience.
- At least 5 years of hands-on data engineering experience building production cloud data pipelines and data warehouse solutions.
- At least 2 years of practical experience integrating automated testing, infrastructure as code, and CI/CD deployment workflows.
- Advanced SQL proficiency and hands-on BigQuery experience managing datasets, partitioning, clustering, views, schema updates, access controls, and query optimization.
- Proven experience designing and running production ELT/ETL pipelines, managing state and incremental loading, and scheduling workflows with Cloud Composer or Apache Airflow.
- Strong hands-on programming skills in Python and Bash shell scripting for container and pipeline workflows.
- Practical experience extracting data from enterprise REST or GraphQL APIs, handling pagination, rate limits, OAuth flows, and integration with secret-management systems.
- Hands-on experience developing modular data transformations, dependencies, and automated test assertions using Dataform or comparable SQL-based frameworks.
- Experience writing reusable Terraform modules to provision GCP data and IAM resources, including BigQuery, Cloud Storage, service accounts, Cloud Run, and Cloud Composer.
- Practical experience maintaining GitLab CI/CD pipelines, including environment stages, automated testing, container builds, and branch management.
- Experience implementing automated data-quality controls, reconciliation, observability, alerting, failure recovery, and production support.
- Experience applying access controls, secure credential handling, lineage, auditability, and data-governance requirements within a regulated or highly governed environment.
- Demonstrate Key’s Values and abide by Key’s Code of Conduct.
Nice To Haves
- Experience packaging and running lightweight workloads using Docker, Cloud Build, and Cloud Run.
- Prior experience building reporting data models around software engineering tools such as Jira, ServiceNow, and GitLab.
- Familiarity with event-driven data ingestion using Pub/Sub or Apache Beam / Google Cloud Dataflow.
- Familiarity with engineering, operational, productivity, quality, cost, risk, or AI-related data.
Responsibilities
- Design, build, and support production batch and incremental pipelines using Python, SQL, and Cloud Composer to collect and process data and telemetry from enterprise tools and source systems.
- Implement and maintain BigQuery datasets, tables, views, and transformation workloads; optimize partitioning, clustering, query performance, access, storage, and compute costs.
- Write and deploy automated transformation workflows and data-quality checks using Dataform.
- Create and maintain repeatable Terraform configurations and GitLab CI/CD pipelines to automate testing, credential handling, and deployments across development, test, and production environments.
- Build secure integrations with enterprise APIs and source systems, applying appropriate authentication and credential management, retry, error-handling, and rate-limiting patterns.
- Configure pipeline monitoring, alerting, schema validation, and reconciliation jobs.
- Partner with the Data Product Owner, data modelers, architects, and platform teams to translate product requirements, metric definitions, data-quality expectations, and governance requirements into reliable technical solutions.
- Maintain technical documentation, lineage, data contracts, implementation guides, and evidence supporting governance and audit requirements.
- Conduct code reviews, establish reusable coding templates, and mentor team engineers on data engineering and version-control best practices.
Benefits
- Eligible for incentive compensation, which may include production, commission, and/or discretionary incentives.
- Eligible for company benefits; specific benefits are referenced through a benefits list linked in the posting.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
