Senior Data Engineer - Data Engineering

Plaid • San Francisco, CA

1d

About The Position

We believe that the way people interact with their finances will drastically improve in the next few years. We’re dedicated to empowering this transformation by building the tools and experiences that thousands of developers use to create their own products. Plaid powers the tools millions of people rely on to live a healthier financial life. We work with thousands of companies like Venmo, SoFi, several of the Fortune 500, and many of the largest banks to make it easy for people to connect their financial accounts to the apps and services they want to use. Plaid’s network covers 12,000 financial institutions across the US, Canada, UK and Europe. Founded in 2013, the company is headquartered in San Francisco with offices in New York, Washington D.C., London and Amsterdam. Making data-driven decisions is key to Plaid's culture. To support that, we need to scale our data systems while maintaining correct and complete data. We provide golden datasets and tooling to teams across engineering, product, and business and help them explore our data quickly and safely to get the data insights they need, which ultimately helps Plaid serve our customers more effectively. In addition, Plaid will not be successful if we can't move quickly. We build the data systems and tools that enable everyone at Plaid to be data-driven, making analytics easy, obvious, and proactive across the company. Data Engineers heavily leverage SQL and Python to build data workflows that integrate with our Golang applications. We use tools like DBT, Airflow, Redshift, Atlan, and Retool to orchestrate data pipelines and define workflows. We work with engineers, product managers, business intelligence, data analysts, and many other teams to build Plaid's data strategy and a data-first mindset. You will be in a high impact role that will directly enable business leaders to make faster and more informed business judgements based on the datasets you build. You will have the opportunity to carve out the ownership and scope of internal datasets and visualizations across Plaid which is a currently unowned area that we intend to take over and build SLAs on. You will have the opportunity to learn best practices and up-level your technical skills from our strong DE team and from the broader Data Platform team. You will collaborate with and have strong and cross functional partnerships with literally all teams at Plaid from Engineering to Product to Marketing/Finance etc.

Requirements

• 4+ years of dedicated data engineering experience, solving complex data pipelines issues at scale.
• You’ve have experience building data models and data pipelines on top of large datasets (in the order of 500TB to petabytes)
• You value SQL as a flexible and extensible tool, and are comfortable with modern SQL data orchestration tools like DBT, Mode, and Airflow.
• You have experience working with different performant warehouses and data lakes; Redshift, Snowflake, Databricks.
• You have experience building and maintaining batch and realtime pipelines using technologies like Spark, Kafka.
• You appreciate the importance of schema design, and can evolve an analytics schema on top of unstructured data.
• You are excited to try out new technologies.
• You like to produce proof-of-concepts that balance technical advancement and user experience and adoption.
• You like to get deep in the weeds to manage, deploy, and improve low level data infrastructure.
• You are empathetic working with stakeholders. You listen to them, ask the right questions, and collaboratively come up with the best solutions for their needs while balancing infra and business needs.
• You are a champion for data privacy and integrity, and always act in the best interest of consumers.

Responsibilities

• Understanding different aspects of the Plaid product and strategy to inform golden dataset choices, design and data usage principles.
• Have data quality and performance top of mind while designing datasets
• Leading key data engineering projects that drive collaboration across the company.
• Advocating for adopting industry tools and practices at the right time
• Owning core SQL and python data pipelines that power our data lake and data warehouse.
• Well-documented data with defined dataset quality, uptime, and usefulness.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed