About The Position

The Lead Data & AI Engineer – Databricks Data Analytics & Agent Bricks is responsible for designing, building, governing, and operating enterprise-grade data engineering and AI-agent solutions on the Databricks Lakehouse Platform and Databricks Agent Bricks / Mosaic AI Agent Framework. The role ensures data products, AI/BI dashboards, Genie spaces, RAG and Agent Bricks solutions, and analytical workloads use secure, scalable, compliant, cost-aware, and reusable engineering patterns aligned with Orbia’s Lakehouse, governance, and Responsible AI standards. The position provides hands-on technical leadership to data and analytics engineers and partners with business groups, platform owners, security, infrastructure, integration, governance, and external partners. It translates business and AI use cases into production-ready Databricks solutions, establishes architecture and engineering standards, oversees deployment and operations, and leads and mentors a small team of data and analytics engineers. Orbia Advance Corporation is a purpose-led company operating in 41 countries with more than 22,000 employees. Its business groups provide solutions across areas including building and infrastructure, data communications, chemicals, water and climate solutions, and precision irrigation. Orbia describes its purpose as advancing life around the world and addressing issues such as rapid urbanisation, water and food scarcity, and a growing and aging population.

Requirements

  • 12–14 years of hands-on experience with the Databricks Lakehouse Platform, including Apache Spark, PySpark, Delta Lake, Unity Catalog, Delta Live Tables, Workflows, SQL Warehouses, MLflow, and Model Serving.
  • At least 2 years of hands-on or strong architecture experience with Databricks Agent Bricks / Mosaic AI Agent Framework, Vector Search, Genie spaces, AI/BI Dashboards, and RAG-based agentic applications grounded on Lakehouse data.
  • Strong programming skills in Python, PySpark, and advanced SQL, with deep understanding of distributed computing, performance tuning, partitioning, Z-Ordering, file optimization, and Delta best practices.
  • Experience with Databricks CI/CD using Databricks Asset Bundles, Azure DevOps and/or GitHub Actions, infrastructure-as-code such as Terraform, and DEV/QA/PROD environment promotion patterns.
  • Experience with the Azure data ecosystem, including ADLS Gen2, Azure Data Factory, Azure Key Vault, Entra ID, Microsoft Purview, Azure Monitor, Azure API Management, and Azure OpenAI services.
  • Experience designing and operating ingestion pipelines from SAP (ECC/S4 HANA, BW, BTP), SharePoint, ServiceNow, Coupa, Salesforce, REST APIs, files, and streaming sources using Auto Loader, ADF, and partner connectors.
  • Experience designing RAG and agent solutions using enterprise documents, semantic indexes, embeddings, vector search, citations, chunking, grounding evidence, and answer evaluation.
  • Experience designing secure enterprise integrations using REST APIs, OAuth 2.0, managed identities, service principals, APIM, event-driven patterns, SAP BTP, ServiceNow APIs, or Microsoft Graph APIs.
  • Strong understanding of governance controls, including Unity Catalog, row- and column-level security, ABAC, RBAC, least privilege, segregation of duties, access reviews, private endpoints, audit logging, DLP, Purview, data classification, lineage, and SOX/ITGC expectations.
  • Experience in AI/data governance, Responsible AI, model and agent evaluation, prompt engineering, hallucination control, content safety, human approval gates, and production release criteria.
  • Demonstrated experience leading a team of data or analytics engineers, including hands-on technical leadership, mentoring, code reviews, sprint planning, and performance development.
  • Experience preparing architecture review documents, solution and low-level design documents, technical standards, operating procedures, decision records, and architecture review board materials.
  • Strong understanding of cost management for Databricks and AI solutions, including DBU usage, SQL Warehouse sizing, storage, vector search cost, model endpoints, monitoring, and FinOps guardrails.
  • Experience working in an international environment at a global company with cross-functional teams and business stakeholders across multiple regions.
  • Excellent verbal and written communication skills in English.
  • Ability to build trustworthy relationships with business users, platform teams, security, architecture, vendors, and implementation partners.
  • Self-starter able to provide technical leadership, mobilize business partners, analyze requirements, challenge solution assumptions, push back when required, and remain result-oriented.

Responsibilities

  • Define and govern Databricks Lakehouse and Agent Bricks architecture principles, design standards, and reusable reference patterns for enterprise data and AI-agent solutions.
  • Design end-to-end data engineering patterns for ingestion, bronze/silver/gold medallion architecture, Delta Live Tables, streaming, batch processing, and data product publishing on Unity Catalog.
  • Design Agent Bricks / Mosaic AI Agent Framework patterns for analytical agents, RAG applications, Genie spaces, AI/BI dashboards, and business assistants grounded in governed Lakehouse data.
  • Translate approved business and AI use cases into solution design documents, data contracts, platform-fit decisions, integration patterns, and production-readiness plans.
  • Define model and tool selection guidance covering accuracy, latency, cost, privacy, security, data sensitivity, and business purpose.
  • Establish engineering guardrails for prompt engineering, RAG grounding, tool calling, human-in-the-loop approvals, write-back controls, and auditability.
  • Support AI/data governance, architecture review board submissions, decision records, architecture exceptions, and platform standards for Databricks solutions.
  • Partner with Data, Analytics & AI leadership to align the Databricks engineering roadmap with Orbia’s enablement strategy and scalable operating model.
  • Evaluate Databricks capabilities for enterprise fit, including Unity Catalog, Delta Live Tables, Workflows, SQL Warehouses, MLflow, Model Serving, Vector Search, Genie, AI/BI Dashboards, and Agent Bricks.
  • Collaborate with cloud, infrastructure, network, security, integration, and AI architecture teams to design DEV, QA, and PROD deployment patterns using Databricks Asset Bundles and CI/CD.
  • Define secure ingestion and integration patterns from enterprise systems and sources, including SAP, SharePoint, ServiceNow, Coupa, Salesforce, REST APIs, file shares, and event sources.
  • Design API-led integration patterns using Azure API Management, Entra ID, OAuth, managed identities, service principals, and approved enterprise gateways for Databricks-hosted agents and endpoints.
  • Lead technical design reviews for data pipelines, Unity Catalog models, agent tools, vector indexes, model endpoints, evaluation workflows, monitoring, and release gates.
  • Build and review production-grade PySpark, Python, and SQL code; enforce coding standards, peer reviews, unit and integration testing, and automated quality checks.
  • Support partner and vendor technical reviews to validate delivery approach, platform compatibility, security posture, and architecture compliance.
  • Define operational runbooks for Databricks workspaces, Unity Catalog, jobs, DLT pipelines, Agent Bricks lifecycle, prompt/version control, evaluations, monitoring, telemetry, and incident support.
  • Ensure solutions follow least-privilege RBAC/ABAC, access reviews, environment segregation, audit logging, data protection, lineage, and Responsible AI controls.
  • Provide technical leadership, mentoring, and guidance to data engineers, analytics engineers, platform administrators, business analysts, and implementation partners.
  • Monitor and optimize DBU consumption, storage, SQL Warehouse usage, vector search cost, model endpoint usage, latency, quality, security, and scalability.
  • Review delivered solutions against engineering principles, governance expectations, SOX/ITGC controls, and production acceptance criteria before go-live.
  • Continuously improve Databricks engineering standards based on lessons learned, platform roadmap changes, business adoption, and operational feedback.
  • Lead and mentor a small team of data and analytics engineers, including code reviews, sprint planning, capacity planning, performance feedback, and skill development.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service