Databricks Engineer

Booz Allen Hamilton • Atlanta, GA

1d • $86,800 - $198,000

About The Position

As a Databricks Engineer, you will design, build, and operate Databricks Lakehouse solutions in the Azure ecosystem. You will develop and support Azure Data Factory pipelines that promote data from enterprise applications through bronze, silver, and gold medallion layers. Working with architects, analysts, developers, engineering teams, clients, and mission stakeholders, you will investigate pipeline and data-quality issues, recommend Lakehouse patterns, translate requirements into working solutions, and help determine how data is ingested, refined, and published for analytics.

The role uses Databricks and Azure best practices, including Azure Data Factory, Azure storage and identity services, Apache Spark, PySpark, SQL, Delta Lake, and Unity Catalog. You will design data models, build reliable batch and incremental ETL/ELT pipelines, and deliver analytics-ready datasets. You will also develop and support Databricks analytics, including dashboards, Genie agents, and data products used by internal teams, external clients, or downstream applications. You will help design, develop, and test cloud data components and support their operation.

Requirements

• 5 or more years of experience building and operating production data platforms on Databricks.
• Experience designing and implementing Lakehouse architectures using Delta Lake, Apache Spark (such as PySpark or Spark SQL), and bronze, silver, and gold medallion patterns.
• Experience developing and supporting Azure Data Factory pipelines that ingest and promote datasets from enterprise applications into Databricks.
• Knowledge of the Azure data ecosystem, including Data Factory, Azure storage, identity and access, and integration with Databricks.
• Knowledge of Unity Catalog, data governance, and access control on Databricks.
• Ability to design, develop, troubleshoot, and optimize batch and incremental ETL/ELT pipelines across source systems and Lakehouse layers.
• Ability to develop and support Databricks analytics, including SQL and Spark datasets, dashboards, and Genie agents.
• Ability to collaborate with architects, developers, and mission stakeholders and translate requirements into Azure Databricks solutions.
• Public Trust determination is required.
• Bachelor’s degree in Computer Science, Information Systems, or Engineering.

Nice To Haves

• Experience with Databricks Asset Bundles, workspace CI/CD, and Git-based promotion across development, test, and production.
• Experience integrating Databricks with enterprise source systems such as Azure SQL or SQL Server.
• Experience building Databricks dashboards and configuring Genie spaces or agents for business-user analytics.
• Knowledge of schema evolution, MERGE-based incremental loads, liquid clustering or Z-ordering, and performance tuning on large Delta tables.
• Ability to define reusable pipeline patterns, coding standards, and operational practices for a multi-table Lakehouse.
• Databricks Certified Data Engineer Associate, Databricks Certified Data Engineer Professional, or Databricks Certified Platform Administrator certification.

Responsibilities

• Design, build, and operate Databricks Lakehouse solutions in the Azure ecosystem.
• Develop and support Azure Data Factory pipelines that ingest and promote enterprise application datasets into Databricks bronze, silver, and gold layers.
• Design data models and reliable batch and incremental ETL/ELT pipelines using Azure and Databricks services and tools.
• Troubleshoot pipeline and data-quality issues, recommend Lakehouse patterns, and help determine how source data should be landed, refined, and published.
• Develop and support Databricks analytics, including SQL and Spark datasets, dashboards, Genie agents, and data products.
• Design, develop, and test cloud data components and deliver analytics-ready datasets.
• Collaborate with architects, analysts, developers, clients, engineering teams, and mission stakeholders to translate requirements into working solutions.

Benefits

• Health, life, disability, financial, and retirement benefits.
• Paid leave.
• Professional development.
• Tuition assistance.
• Work-life programs.
• Dependent care.
• Recognition awards for exceptional performance and demonstration of company values.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level