Lead Data Engineer

Wells Fargo & Company•Irving, TX

1d•Hybrid

About The Position

Wells Fargo is seeking a Lead Data Engineer – Data & Insights to design, build, and support scalable, secure, and high-performance data platforms leveraging Google Cloud Platform (GCP) and enterprise-grade data technologies. This role will lead strategic initiatives involving Apache Spark, Apache Kafka, BigQuery, and MS SQL Server to enable real-time and batch data processing, advanced analytics, and regulatory reporting while aligning with enterprise risk, governance, and compliance standards. Learn more about the career areas and lines of business at wellsfargojobs.com.

Requirements

5+ years of Software Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education
3+ years of experience leading in Data Engineering, Data Management, SQL Server, MongoDB, Teradata, ETL
3+ years hands-on experience with GCP (Google Cloud Platform) including GCP BigLake, BigQuery, and Dataplex
3+ years building and maintaining data pipelines using Apache Spark (Java SDK), Apache Iceberg, Kafka, and dbt+
3+ years of experience working with large-scale, distributed data systems and platforms such as Cloudera Data Platform.
3+ years of experience with Agile, Autosys, CI/CD, and DevOps practices using tools such as Jenkins, GitLab, or Tekton

Nice To Haves

Experience in converting SSIS packages to Apache Spark or other GCP-native data processing frameworks, modernizing legacy workflows
Strong experience in data engineering, APIs, and connectivity solutions (SQL, NoSQL, REST, GraphQL)
Strong background in building and maintaining Approved Data Stores (ADS) and Operational Data Stores (ODS) to support enterprise analytics and operational requirements.
3+ years of Merchant Services or Credit Card industry experience
Familiarity with data governance, security, and privacy standards, including lineage tracking and auditability in cloud environments.
Strong ability to document projects and processes to ensure clear understanding and replicability

Responsibilities

Lead the design and development of large-scale data pipelines using Apache Spark (batch & streaming) and GCP-native services.
Work with a high-performing team of data engineers, product managers, and data scientists to drive the development and delivery of data products in the Merchant domain.
Architect and implement event-driven data platforms using Apache Kafka and Pub/Sub for real-time ingestion and processing.
Build and optimize data warehouse and lakehouse architectures using BigQuery, Cloud Storage, and MS SQL Server.
Design and implement real-time streaming solutions using Kafka Streams, Spark Structured Streaming, or Dataflow.
Optimize BigQuery performance and cost efficiency using partitioning, clustering, and workload management best practices.
Ensure data quality, lineage, observability, and governance through enterprise frameworks and controls.
Collaborate with cross-functional teams to deliver secure, compliant, and scalable data solutions.
Provide leadership in distributed systems design, event streaming architecture, and enterprise data management.
Support incident management, root cause analysis, and continuous improvement for data services.
Act as a key participant in developing standards and companywide best practices for engineering complex and large-scale technology solutions for technology engineering disciplines
Design, code, test, debug, and document for projects and programs
Make decisions in developing standard and companywide best practices for engineering and technology solutions requiring understanding of industry best practices and new technologies, influencing and leading technology team to meet deliverables and drive new initiatives
Collaborate and consult with key technical experts, senior technology team, and external industry groups to resolve complex technical issues and achieve goals
Lead projects, teams, or serve as a peer mentor