Sr Data Engineer

Disney•San Francisco, CA

38d

About The Position

Disney Entertainment and ESPN Product & Technology Technology is at the heart of Disney's past, present, and future. Disney Entertainment and ESPN Product & Technology is a global organization of engineers, product developers, designers, technologists, data scientists, and more - all working to build and advance the technological backbone for Disney's media business globally. The team marries technology with creativity to build world-class products, enhance storytelling, and drive velocity, innovation, and scalability for our businesses. We are Storytellers and Innovators. Creators and Builders. Entertainers and Engineers. We work with every part of The Walt Disney Company's media portfolio to advance the technological foundation and consumer media touch points serving millions of people around the world. Here are a few reasons why we think you'd love working here: 1. Building the future of Disney's media: Our Technologists are designing and building the products and platforms that will power our media, advertising, and distribution businesses for years to come. 2. Reach, Scale & Impact: More than ever, Disney's technology and products serve as a signature doorway for fans' connections with the company's brands and stories. Disney+. Hulu. ESPN. ABC. ABC News…and many more. These products and brands - and the unmatched stories, storytellers, and events they carry - matter to millions of people globally. 3. Innovation: We develop and implement groundbreaking products and techniques that shape industry norms and solve complex and distinctive technical problems. Job Summary: Product Engineering is a unified team responsible for the engineering of Disney Entertainment & ESPN digital and streaming products and platforms. This includes product engineering, media engineering, quality assurance, engineering behind personalization, commerce, lifecycle, and identity. This role involves building and maintaining the Audience Manager Platform (AMP), a centralized Disney audience data platform that captures & processes all Disney consumers cross channel behavior and delivers varying summarizations in a timely manner to multiple endpoints to support all business needs. At its core, it is a product that provides a comprehensive source of truth for customer data. It is built around customer profiles and is able to provide insights from across the business so stakeholders can quickly act on the data through activation pipelines. The role requires expert knowledge of building scalable, fault-tolerant data processing pipelines and backend to ensure the reliable delivery of audience data.

Requirements

5+ years of big data engineering experience modeling and developing large data pipelines
Hands-on experience with distributed systems such as Spark, Hadoop (HDFS, Hive, Presto, PySpark) to query and process data
Strong Python and SQL skills processing big datasets
Experience with at least one major MPP or cloud database technology (Snowflake, Redshift, Big Query, Databricks)
Familiarity with Data Modeling techniques and Data Warehousing standard methodologies and practices
API and backend development experience
Bachelor's degree in Computer Science, Information Systems, Software, Electrical or Electronics Engineering, or comparable field of study, and/or equivalent work experience.

Responsibilities

Build and maintain audience platform initiatives to improve and expand audience targeting capabilities.
Partner with technical and non-technical colleagues to understand audience data and targeting requirements.
Work with engineering teams to collect required data from internal and external systems.
Support Agile methodologies such as Scrum by actively participating in regular ceremonies such as stand-up, retrospectives and sprint planning.
Design table structures and define ETL pipelines to build performant and scalable solutions
Develop Data Quality checks
Participate in on-call support
Develop and maintain ETL routines using ETL and orchestration tools such as Airflow
Perform ad hoc analysis and SQL and ETL tuning as necessary.
Participate in code reviews and problem-solving.