Senior Data Engineer

Credit Karma•Charlotte, NC

11d

About The Position

Intuit Credit Karma is a mission-driven company, focused on championing financial progress for our more than 140 million members globally. While we're best known for pioneering free credit scores, our members turn to us for everything related to their financial goals, including identity monitoring, applying for credit cards, shopping for insurance and loans (car, home and personal) and savings accounts and checking accounts – all for free. Credit Karma has grown significantly through the years: we now have more than 1,700 employees across our offices in Oakland, Charlotte, Culver City, San Diego, London, Bangalore, and New York City. Banking services provided by MVB Bank, Inc., Member FDIC The Lightbox Engineering team is responsible for enabling our partners to build and host their proprietary models on Credit Karma. We provide various applications and data, hosted on Google Cloud Platform, which help our partners to build their models, analyze performance and iterate. As a Data Engineer IV, you will own engineering efforts to provide streaming and batch data to our partners with low latency and high data quality. What’s great about the role: You will have the opportunity to contribute greatly to an extremely engineering-focused organization. You will get broad exposure to our full stack, consisting of technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform, Google Cloud Platform, etc. You will get a lot of opportunities to learn and grow in building fast, scalable and secure data pipelines and infrastructure. Your contributions will have a noticeable impact on our partners, members as well as your fellow Karmanauts (that’s what we call ourselves). You will grow and learn and have fun doing it--it’s part of our culture. And, of course, all those awesome company perks that you have probably already read about.

Requirements

5+ years of significant professional experience in server-side development written in Scala, Java, Go, or Python
5+ years of professional experience developing large-scale, high-volume services
5+ years of experience managing data pipelines and infrastructure using Dataflow, Spark, Hadoop, Flink, and Airflow.
Professional experience across the entire software development process, from development environment, to CI & CD including CircleCi, Jenkins, or TravisCi, to production deployment, product and application performance measurement
Confident articulating complex technical content to peers and partners across the organization via Technical Design Documents.
Proven ability and interest in mentorship of junior engineers.

Nice To Haves

DevOps experience with Google Cloud Platform, AWS, or Azure
Data quality experience especially using & improving internal tools to automatically detect issues.
Experience building data pipelines that deal with sensitive information
Have owned a part of or entire data pipeline at previous position

Responsibilities

Build scalable data pipelines leveraging Cloud Composer and Airflow.
Work on critical efforts designing and building our next generation streaming data pipeline, leveraging PubSub, Scala, Dataflow, Spark and Google Cloud Platform.
Design, build and launch extremely efficient & reliable data warehouse solutions and data pipelines to make large volumes of data usable for our partners to build models.
Build critical data quality checks to ensure that the data we provide to our partners meets security and compliance requirements.
Participate in code and technical design reviews
Take an active role in the technical growth of other team members and the processes that support overall team growth
Scope and plan work across multi-team efforts, working with collaborators to establish a product vision in technical terms, and plan implementation and delivery