Principal Data Engineering Lead

Visa•Bellevue, WA

6d•Hybrid

About The Position

The Principal Data Engineering Lead is a senior technical leader responsible for guiding the design, development, and optimization of Visa's large‑scale data platforms and cloud-based analytics environments. This role provides architectural direction, leads complex engineering initiatives, and mentors teams while remaining deeply hands-on with modern data technologies. The Lead Data Engineer drives technical best practices, ensures platform scalability, and influences data engineering strategy for key products and business domains. Responsibilities: Lead the architecture and delivery of large-scale, high-performance data pipelines and processing frameworks across Hadoop and multi-cloud environments.Design scalable data models, lakehouse structures, and distributed data processing solutions that support analytics, machine learning, and real-time data needs.Provide technical leadership to Senior and Staff Data Engineers, conducting design reviews, guiding implementation decisions, and ensuring engineering excellence.Partner with cross-functional teams to translate business and product requirements into robust technical designs and data solutions.Develop and improve engineering best practices for data governance, quality, observability, testing, and cloud resource optimization.Drive adoption of cloud-native data technologies, automation frameworks, and reusable components that improve development velocity and system reliability.Lead complex data modernization efforts, including cloud migration, data lake/lakehouse consolidation, and performance optimization of critical pipelines.Evaluate new tools and technologies, influencing platform evolution within the scope of assigned domains or product areas.Collaborate with product, analytics, and platform teams to ensure alignment on data strategy and architectural roadmaps.Mentor engineers at all levels, providing technical coaching and fostering a culture of continuous improvement. This is a hybrid position. Expectation of days in the office will be confirmed by your Hiring Manager.

Requirements

10+ years of relevant work experience with a Bachelor's Degree or at least 7 years of work experience with an Advanced degree (e.g. Masters, MBA, JD, MD) or 4 years of work experience with a PhD, OR 13+ years of relevant work experience.
Advanced expertise in building and optimizing large-scale distributed data systems using Hadoop, Spark, and modern lakehouse architectures.
Strong programming proficiency in PySpark, Scala, and Python with experience implementing scalable, production-grade data applications.
Deep experience designing and tuning RDBMS, NoSQL, and distributed SQL systems.
Mastery of SQL and distributed query engines such as Presto, Trino, Hive, and SparkSQL.
Strong knowledge of data modeling, ETL/ELT design, and data warehousing methodologies.
Proven experience architecting and operating data solutions on AWS, GCP, and Azure, including cloud data lakes, orchestration tools, and cost-effective storage/compute designs.
Advanced proficiency in Databricks, including:
Expert in building and optimizing notebooks and production jobs
Expert in Delta Lake design and optimization
Expert in Cluster configuration and workspace administration
Expert in CI/CD integration for data workloads
Expert in performance tuning for large distributed jobs
Expert in demonstrated ability to lead technical initiatives, communicate architectural decisions, and influence engineering direction across multiple teams.
Strong problem-solving skills with the ability to troubleshoot complex data and performance issues.

Nice To Haves

12 or more years of relevant work experience with a Bachelor's Degree or at least 7 or more years of work experience with an Advanced degree (e.g. Masters, MBA, JD, MD) or 4 years of work experience with a PhD, OR 13 or more years of relevant work experience.

Responsibilities

Lead the architecture and delivery of large-scale, high-performance data pipelines and processing frameworks across Hadoop and multi-cloud environments.
Design scalable data models, lakehouse structures, and distributed data processing solutions that support analytics, machine learning, and real-time data needs.
Provide technical leadership to Senior and Staff Data Engineers, conducting design reviews, guiding implementation decisions, and ensuring engineering excellence.
Partner with cross-functional teams to translate business and product requirements into robust technical designs and data solutions.
Develop and improve engineering best practices for data governance, quality, observability, testing, and cloud resource optimization.
Drive adoption of cloud-native data technologies, automation frameworks, and reusable components that improve development velocity and system reliability.
Lead complex data modernization efforts, including cloud migration, data lake/lakehouse consolidation, and performance optimization of critical pipelines.
Evaluate new tools and technologies, influencing platform evolution within the scope of assigned domains or product areas.
Collaborate with product, analytics, and platform teams to ensure alignment on data strategy and architectural roadmaps.
Mentor engineers at all levels, providing technical coaching and fostering a culture of continuous improvement.