Lead Data Engineer - Experimentation Platform - 1633

aKube • Santa Monica, CA

2d • Hybrid

About The Position

This role involves designing and building scalable data platforms to support experimentation and A/B testing. The Lead Data Engineer will develop batch and streaming data pipelines for large-scale user and product datasets, create reusable datasets and frameworks for experimentation, analytics, and product measurement, and design dimensional data models and analytics-ready data products. The position also requires implementing automated data quality, validation, monitoring, lineage, and governance, as well as building production-grade deployment pipelines with CI/CD and observability. The engineer will collaborate with Product, Engineering, Data Science, and Analytics teams to deliver scalable data solutions and optimize data infrastructure for experimentation, personalization, and machine learning workloads. Mentoring engineers and establishing best practices for large-scale data engineering are also key responsibilities.

Requirements

• Python
• SQL
• Data Engineering
• ETL / ELT
• Apache Spark
• Databricks
• Snowflake
• Apache Kafka
• Apache Airflow
• Streaming Data Pipelines
• Data Modeling
• Data Warehousing / Lakehouse
• A/B Testing / Experimentation Platforms
• CI/CD for Data Pipelines
• Data Quality & Data Governance
• Cloud Data Platforms
• Bachelor's degree in Computer Science, Engineering, Mathematics, Statistics, or a related technical field.
• 7+ years of experience in data engineering or large-scale data platforms.
• Strong experience with distributed data processing and cloud-based data architectures.
• Hands-on experience with Python, SQL, Spark, Databricks, Snowflake, Kafka, and Airflow.
• Strong understanding of data modeling, ETL/ELT, streaming architectures, and lakehouse concepts.
• Experience building experimentation, analytics, personalization, or ML data platforms.
• Experience implementing CI/CD, automated testing, monitoring, and data governance.
• Strong system design and architecture experience.
• Experience mentoring engineers and leading technical initiatives.

Nice To Haves

• Experimentation platforms or A/B testing infrastructure.
• Causal inference or product analytics experience.
• ML feature engineering and model lifecycle pipelines.
• Infrastructure automation and observability.
• Subscription, streaming media, advertising, or consumer product experience.
• MS or PhD in a related technical field.

Responsibilities

• Design and build scalable data platforms supporting experimentation and A/B testing.
• Develop batch and streaming data pipelines for large-scale user and product datasets.
• Build reusable datasets and frameworks for experimentation, analytics, and product measurement.
• Design dimensional data models and analytics-ready data products.
• Implement automated data quality, validation, monitoring, lineage, and governance.
• Build production-grade deployment pipelines with CI/CD and observability.
• Partner with Product, Engineering, Data Science, and Analytics teams to deliver scalable data solutions.
• Optimize data infrastructure supporting experimentation, personalization, and machine learning workloads.
• Mentor engineers and establish best practices for large-scale data engineering.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior