Senior Director, Data Engineering

Albertsons Companies • Pleasanton, CA

8d • $218,900 - $297,380 • Onsite

About The Position

The Tech & AI Department has an opening for a Senior Director, Data Engineering. This position is located in Pleasanton, California. Central to this mission is democratizing access to data — enabling customer experience innovation, predictive models, and business intelligence by partnering closely with technology teams, data scientists, data analysts, and business users across the enterprise. This role owns the architecture, the team, and the outcomes. The leader sets technical direction for how data is ingested, modeled, governed, and consumed — and deploys agentic AI to make the engineering organization itself faster and more scalable. This means owning a multi-year roadmap, a technology budget, and a team of 50+ engineers working across three areas: data platform, data analytics, and intelligence platform.

Requirements

• Bachelor’s degree in computer science, engineering, or a related quantitative discipline - required.
• 15 plus years in data engineering, data architecture, or closely related technical disciplines.
• 8 plus years in building and leading data engineering teams
• 5 plus years in senior leadership with direct accountability for large-scale data engineering organizations (25+ engineers) in a complex enterprise environment.
• Proven track record building and scaling production-grade data platforms at Fortune 500 scale.
• Experience owning technology budgets.
• Demonstrated success leading organizations through technology modernization and methodology shifts.
• Expert-level command of data engineering fundamentals — ELT/ETL, data integration, cataloging, wrangling, quality, governance, and lineage — with proven ability to select and implement the right tooling at enterprise scale.
• Expert-level command of cloud data platforms; strong preference for GCP (BigQuery, Dataflow, Pub/Sub, Composer)
• Deep expertise in data lakehouse, Medallion architecture, and large-scale warehouse engineering
• Deep understanding of data mesh principles, semantic layer design, and conversational AI architecture
• Proficient in Airflow, Kafka/Flink, Python, SQL, and Apache Spark
• Strong grasp of data analytics, BI tooling, and self-service reporting platforms (e.g., Power BI, Looker).
• Knowledge of MLOps tooling, feature stores, vector databases, and LLM API integration for production AI/ML systems
• Proficiency in leveraging AI tools for daily engineering tasks to enhance productivity, optimize effort, and ensure cost-aware AI assistance.
• Familiarity with Retrieval Augmented Generation (RAG) architectures, including semantic layers for data retrieval and grounding Large Language Model (LLM) responses.
• Conceptual understanding of LLM-based applications (e.g., chatbots, Q&A systems), encompassing prompt engineering, context management, and response generation.
• Exposure to agentic and multi-agent AI systems, including agent roles, tool utilization, memory management, and workflow orchestration.
• Practical experience with LLM application frameworks (e.g., LangChain, LangGraph, or Google GenAI tools) for prototyping and integrating AI-driven solutions.

Nice To Haves

• Master's degree or MBA - strongly preferred.
• Experience in retail, CPG, grocery, or a similarly complex, high-velocity transactional industry — strongly preferred.
• Familiarity with retail-specific data domains: supply chain, merchandising, loyalty, store operations, and pharmacy.

Responsibilities

• Owns the multi-year data platform roadmap and architectural evolution across lakehouse, Medallion, and streaming infrastructure — driving high-consequence technology decisions with full accountability for ROI, budget, team goals, KPIs, and delivery milestones.
• Defines the strategy and operating model for embedding agentic AI — spanning code generation, automated testing, pipeline monitoring, and auto-remediation — across the engineering function to drive measurable gains in velocity and reliability.
• Enforces architectural standards for trusted, governed, and discoverable data — driving data mesh adoption, data contracts, and domain ownership — while overseeing feature stores, vector databases, and RAG pipelines that power production AI.
• Delivers a governed conversational and semantic access layer across core retail functions — like merchandising, supply chain, store operations — into a unified, queryable knowledge layer with governance guardrails ensuring trustworthy, business-ready outputs.
• Adopts and enforces enterprise governance frameworks, quality standards, and observability practices — ensuring compliance across all data engineering workloads, proactively recommending improvements, and partnering with Audit, Compliance, and Governance leaders on data security and regulatory adherence.
• Embeds DataOps and platform engineering best practices — CI/CD for data, SLA/SLO ownership, and cloud cost optimization across GCP and Azure — to build a high-throughput, production-grade engineering organization.
• Leads 50+ engineers through directors and senior managers — owning hiring, performance management, career development, and succession — while building psychological safety and developing managers who grow the next generation of engineering leaders.
• Partners with Data Science, ML Engineering, Analytics, Product, and business domains to align data strategy with enterprise priorities — communicating trade-offs and platform direction to C-suite and VP stakeholders with clarity and conviction.
• Partners with business stakeholders to deeply understand the underlying problem before delivering — translating business intent into governed BI, self-service analytics via PowerBI, and API-driven data products that go beyond the ask and drive measurable outcomes.

Benefits

• Competitive wages paid weekly
• Access to up to 50% of your earned wages before payday, via our partnership with Stream
• Associate discounts
• Health and financial well-being benefits for eligible associates (Medical, Dental, 401k and more!)
• Time off (vacation, holidays, sick pay).
• Leaders invested in your training, career growth and development
• An inclusive work environment with talented colleagues who reflect the communities we serve

What This Job Offers

• Job Type: Full-time
• Career Level: Senior