Senior Data Engineer

JLL • Westmont, IL

3d • Onsite

About The Position

The Senior Data Engineer designs and builds governed data foundations, pipelines, APIs, and a Databricks semantic layer to make enterprise data trustworthy, AI-ready, and accessible to non-technical colleagues. The role combines hands-on Databricks and PySpark engineering with partnership across Finance and IT to move the organization toward a modern, agile target-state data architecture.

The position builds AI-ready pipelines and governed access foundations for AI agents, establishes data quality, lineage, observability, and evaluation frameworks, and develops production-grade solutions with data science and business teams. It also designs and maintains a governed semantic layer in Unity Catalog, configures Databricks Genie spaces, and validates semantic outputs with Finance, HR, and other business owners. The role is on-site in Westmont, Illinois, and does not provide visa sponsorship.

Requirements

• Hands-on Databricks experience with Delta Lake, Unity Catalog, Spark SQL, PySpark, and Workflows.
• Strong SQL and Python skills.
• Demonstrated experience building a semantic layer in Databricks, such as Unity Catalog views, metric views, and/or Genie configuration.
• Experience building batch and streaming pipelines and lakehouse/medallion data models.
• Experience developing REST APIs using FastAPI or Flask, with OAuth-based authentication and authorization.
• Working knowledge of AI data engineering concepts, including embeddings, vector search, RAG, LLM agents, context engineering, and AI evaluation.
• Experience with data governance, lineage, quality, and security in an enterprise environment.
• Experience with cloud platforms and CI/CD.
• Must be authorized to work in the United States without visa sponsorship.

Nice To Haves

• Azure cloud experience is preferred.
• Experience with Genie or similar text-to-SQL tools, including accuracy benchmarking.
• Experience with MCP or agent frameworks for enterprise data access.

Responsibilities

• Contribute to the design of information infrastructure and data management processes that move the organization toward an agile, robust, AI-ready target-state data architecture.
• Build systems and pipelines to ingest, cleanse, and normalize diverse internal and external datasets, and structure unstructured data for analytical and AI use.
• Partner with colleagues and external professionals to determine requirements, anticipate future needs, and identify opportunities for data development.
• Develop an understanding of data flows across CRM, Broker & Sales, Finance, HR, MDM, ODS, Data Lake, and EDW.
• Deliver data and API solutions, including natural-language and AI-assisted access to governed data, enabling non-technical staff to make data-driven decisions.
• Develop proofs of concept to validate solution proposals and influence platform architects, product managers, and software engineers.
• Design and develop data lake and API solutions for structured and unstructured data, and provide technical guidance to colleagues migrating to modern platforms.
• Build AI-ready pipelines that curate and version data for retrieval, evaluation, and agentic workflows, including document parsing, chunking, embeddings, and vector search for RAG.
• Develop API and MCP-based foundations that give AI agents governed access to enterprise data.
• Implement data quality, lineage, observability, and evaluation frameworks to measure the accuracy of AI-generated answers against trusted data.
• Apply governance and access controls, including row- and column-level security and masking, so AI solutions respect data classification and privacy requirements.
• Partner with data science and business teams to move AI prototypes into monitored, production-grade solutions.
• Design, build, and maintain a governed semantic layer in Unity Catalog, including curated views, certified metric definitions, and centralized business logic.
• Configure and tune Databricks Genie spaces, including instructions, sample queries, entity matching, and naming conventions, to support accurate natural-language querying.
• Establish semantic-layer standards, documentation, and versioning, and validate outputs against source-of-truth systems with Finance, HR, and other business owners.

Benefits

• 401(k) plan with matching company contributions.
• Comprehensive medical, dental, and vision care.
• Paid parental leave at 100% of salary.
• Paid time off and company holidays.
• Early access to earned wages through Daily Pay.
• Supportive culture and comprehensive benefits package prioritizing mental, physical, and emotional health.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed