Data Engineer, AI

Focused Labs•Chicago, IL

12h•Hybrid

About The Position

Focused is seeking a Data Engineer to build the data foundation behind an enterprise AI platform for a global enterprise client. The role focuses on making clean, governed data available for retrieval by agents and LLM applications, primarily using Databricks. The engineer will design data pipelines and lakehouse models for analytics and AI workloads, implement governance and access controls, and ensure data quality and pipeline reliability. AI-focused work includes preparing unstructured data for RAG systems, managing embeddings and vector search, and building datasets to evaluate LLM retrieval quality. The role also involves deploying cloud data infrastructure and applying software engineering practices to data work. The role requires working from the Chicago office three days per week and up to 20% travel within the United States. Focused cannot sponsor or take over sponsorship of employment visas at this time.

Requirements

At least 5 years of experience in data engineering.
Strong hands-on experience with Databricks, Spark, and Delta Lake.
Proficiency in Python and SQL.
Experience building production data pipelines in a cloud environment; Azure or GCP experience is preferred.
Experience with data modeling and data quality practices.

Nice To Haves

Experience building data pipelines for RAG, embeddings, or vector search.
Experience with Unity Catalog and Mosaic AI.
Experience with streaming technologies such as Structured Streaming, Kafka, or Event Hubs.
Consulting or client-facing experience.
Experience improving an AI system's answers by fixing the underlying data.
Experience handling messy unstructured data at scale.
Databricks certifications or open-source contributions.

Responsibilities

Design and build batch and streaming pipelines in Databricks using Spark, Delta Lake, and Workflows.
Model data in a lakehouse architecture serving analytics and AI workloads.
Implement governance and access controls with Unity Catalog.
Monitor data quality and pipeline reliability.
Build ingestion and chunking pipelines for unstructured data, including documents, PDFs, and knowledge bases, to feed RAG systems.
Generate, store, and refresh embeddings, and operate vector search using Databricks Vector Search or similar tools.
Build datasets for LLM evaluation and help teams measure retrieval quality.
Partner with AI and platform engineers to ensure agents receive the right data with the right permissions.
Deploy data infrastructure on Azure and/or GCP using infrastructure as code.
Apply software engineering practices to data work, including version control, testing, and CI/CD.