[GOA] Senior Data Engineer (Databricks, Python, AI)

Software Mind

3d•Hybrid

About The Position

Join a forward-thinking technology team building a next-generation Intelligence Layer - a governed, AI-powered analytics platform that delivers trusted, real-time answers to questions for both customers and internal users. You'll work on a cutting-edge Databricks stack within a collaborative engineering team, contributing to a system that directly shapes how AI interfaces with business-critical data.

Requirements

Proven hands-on experience with Databricks (non-negotiable)
Minimum 4 years of relevant data engineering or analytics engineering experience
Strong knowledge of data modelling, Delta Tables, Metric Views, and Genie Agents
Excellent skills in SQL and Python
Experience building solutions using agentic AI methods and patterns
Familiarity with Unity Catalog or similar data governance and lineage frameworks
Track record of delivering reliable, production-grade data pipelines and analytical systems

Nice To Haves

Experience with MLflow for experiment tracking, model/agent evaluation, or observability
Knowledge of multi-agent orchestration patterns in enterprise data environments
Familiarity with Model Context Protocol (MCP) or exposing AI capabilities via APIs
Background in the aviation or travel industry and understanding of flight data concepts

Responsibilities

Design, build, and maintain data pipelines and curated datasets within Databricks Unity Catalog
Develop and document Metric Views defining core business metrics (schedules, on-time performance, bookings, pricing) to ensure consistency across the platform
Configure and tune domain-specific Genie Agents for conversational analytics
Contribute to the multi-agent orchestration architecture, including the Supervisor Agent layer
Implement MLflow tracing and evaluation to monitor agent quality and manage versioning
Support the integration of the intelligence layer with external AI tools via the MCP Server
Collaborate with data and product teams to translate business questions into governed, reusable data definitions