Sr Data Engineer - Databricks

Adastra Corporation•Irvine, CA

1d•Hybrid

About The Position

Adastra is looking for a hands-on Data Engineer to join our team supporting a modern enterprise data platform. You'll design and run Databricks pipelines that turn raw operational, marketing, and retail data into trusted, analytics-ready data products the business relies on every day. Working as an embedded member of the data team, you'll own pipelines end to end on the Databricks Lakehouse Platform. If you love clean data, well-tuned Spark jobs, and shipping solutions that people actually use, this role is for you.

Requirements

Bachelor's degree in Computer Science, Information Technology, Engineering, Mathematics, or a related field
5+ years of data engineering experience building and operating production data pipelines
Hands-on expertise with the Databricks Lakehouse Platform, including Delta Lake, Databricks Workflows, and Unity Catalog
Advanced experience with PySpark and Spark SQL
Strong SQL development and performance tuning skills
Solid Python development skills and experience with modern engineering practices
Experience using Git, CI/CD pipelines, testing frameworks, and code review processes
Experience working with AWS, Azure, or GCP and their core compute and storage services
Strong understanding of data modeling concepts, including dimensional and medallion architectures
Experience designing ELT/ETL solutions and integrating data from databases, APIs, files, and event streams
Strong communication skills and the ability to work independently within a client-facing environment

Nice To Haves

Databricks Certified Data Engineer Associate or Professional certification
Experience with Structured Streaming, Auto Loader, Kafka, or Kinesis
Experience working with marketing or customer data platforms such as Braze
Retail, grocery, or consumer packaged goods (CPG) industry experience
Experience supporting Power BI reporting and self-service analytics initiatives

Responsibilities

Design, build, and maintain scalable batch and streaming pipelines on Databricks using PySpark and Spark SQL within a medallion (Bronze/Silver/Gold) architecture
Ingest and unify data from operational databases, cloud storage, APIs, and marketing platforms such as Braze
Model and optimize Delta Lake tables for performance, reliability, and cost efficiency using techniques such as partitioning, liquid clustering, OPTIMIZE, and VACUUM
Deliver analytics-ready data for reporting on sales, shipments, orders, and other key business metrics, supporting Power BI dashboards and downstream analytics
Orchestrate jobs using Databricks Workflows, managing dependencies, retries, alerting, and SLAs
Apply Unity Catalog for data governance, lineage, and access control
Work to a high engineering standard using Git, CI/CD, testing, code reviews, and clear documentation
Monitor pipeline health, troubleshoot issues, and tune Spark jobs and cluster configurations for speed and cost optimization
Partner with business and technical stakeholders to deliver reliable, high-quality data products