Databricks Engineer

Booz Allen Hamilton•Atlanta, GA

22h•$86,800 - $198,000•Onsite

About The Position

As a Databricks Engineer, you will design, build, and operate Databricks Lakehouse solutions in the Azure ecosystem. You will develop and support Azure Data Factory pipelines that move data from enterprise applications into bronze, silver, and gold layers, and help resolve data-platform and data-quality issues with architects, analysts, and engineering teams. You will recommend Lakehouse patterns and help determine how source data is landed, refined, and published for analytics. The role uses Databricks and Azure best practices, including Azure Data Factory, Azure storage and identity services, Apache Spark, PySpark, SQL, Delta Lake, and Unity Catalog. You will design data models, create reliable ETL/ELT pipelines, and deliver analytics-ready datasets. You will also develop and support Databricks analytics such as dashboards, Genie agents, and data products for external clients or downstream applications, collaborating with clients and developers to meet requirements and deliver solutions.

Requirements

5+ years of experience building and operating production data platforms on Databricks.
Experience designing and implementing Lakehouse architectures using Delta Lake, Apache Spark (such as PySpark or Spark SQL), and bronze, silver, and gold medallion patterns.
Experience developing and supporting Azure Data Factory pipelines that ingest and promote enterprise application datasets into Databricks.
Knowledge of the Azure data ecosystem, including Data Factory, Azure storage, identity and access, and integration with Databricks.
Knowledge of Unity Catalog, data governance, and Databricks access control.
Ability to design, develop, troubleshoot, and optimize batch and incremental ETL/ELT pipelines across source systems and bronze, silver, and gold layers.
Ability to develop and support Databricks analytics, including SQL and Spark datasets, dashboards, and Genie agents for internal teams or external clients.
Ability to collaborate with architects, developers, and mission stakeholders and translate requirements into working Azure Databricks solutions.
Public Trust determination is required; selected applicants will be subject to a government investigation and may need to meet U.S. government client eligibility requirements.
Bachelor’s degree in Computer Science, Information Systems, or Engineering.

Nice To Haves

Experience with Databricks Asset Bundles, workspace CI/CD, and Git-based promotion across development, test, and production.
Experience integrating Databricks with enterprise source systems such as Azure SQL or SQL Server.
Experience building Databricks dashboards and configuring Genie spaces or agents for business-user analytics.
Knowledge of schema evolution, MERGE-based incremental loads, liquid clustering or Z-ordering, and performance tuning on large Delta tables.
Ability to define reusable pipeline patterns, coding standards, and operational practices for a multi-table Lakehouse.
Databricks Certified Data Engineer Associate, Databricks Certified Data Engineer Professional, or Databricks Certified Platform Administrator certification.

Responsibilities

Design, build, and operate Databricks Lakehouse solutions in the Azure ecosystem.
Develop and support Azure Data Factory pipelines that ingest and promote enterprise application datasets into Databricks bronze, silver, and gold layers.
Analyze and troubleshoot pipeline and data-quality problems in collaboration with architects, analysts, and engineering teams.
Recommend Lakehouse patterns and help determine how source data should be landed, refined, and published for analytics.
Apply Databricks and Azure best practices to design data models and build reliable ETL/ELT pipelines.
Develop and support Databricks analytics, including SQL and Spark datasets, dashboards, Genie agents, and data products for clients or downstream applications.
Help design, develop, and test cloud data components.
Collaborate with clients and developers to ensure requirements are met and solutions are delivered.