Databricks Engineer

Booz Allen Hamilton • Atlanta, GA

1d

About The Position

As a Databricks Engineer, you will design, build, and operate Databricks Lakehouse solutions in the Azure ecosystem. You will develop and support Azure Data Factory pipelines that move datasets from enterprise applications into Databricks bronze, silver, and gold medallion layers. You will collaborate with architects, analysts, engineering teams, clients, developers, and mission stakeholders to troubleshoot platform and data-quality issues, recommend Lakehouse patterns, translate requirements into working solutions, and deliver cloud data components that meet requirements.

The role applies Databricks and Azure best practices and uses Azure Data Factory, Azure storage and identity services, Apache Spark, PySpark, SQL, Delta Lake, and Unity Catalog. You will design data models, build reliable batch and incremental ETL/ELT pipelines, and deliver analytics-ready datasets. You will also develop and support Databricks analytics, including dashboards, Genie agents, and data products used by internal teams, external clients, or downstream applications.

Requirements

• 5+ years of experience building and operating production data platforms on Databricks.
• Experience designing and implementing Lakehouse architectures using Delta Lake, Apache Spark (such as PySpark or Spark SQL), and bronze, silver, and gold medallion patterns.
• Experience developing and supporting Azure Data Factory pipelines that ingest and promote datasets from enterprise applications into Databricks.
• Knowledge of the Azure data ecosystem, including Data Factory, Azure storage, identity and access, and integration with Databricks.
• Knowledge of Unity Catalog, data governance, and access control on Databricks.
• Ability to design, develop, troubleshoot, and optimize batch and incremental ETL/ELT pipelines.
• Ability to develop and support Databricks analytics, including SQL and Spark datasets, dashboards, and Genie agents.
• Ability to collaborate with architects, developers, and mission stakeholders and translate requirements into Azure Databricks solutions.
• Public Trust determination is required; selected applicants are subject to a government investigation and may need to meet U.S. government client eligibility requirements.
• Bachelor’s degree in Computer Science, Information Systems, or Engineering.

Nice To Haves

• Experience with Databricks Asset Bundles, workspace CI/CD, and Git-based promotion across development, test, and production.
• Experience integrating Databricks with enterprise source systems such as Azure SQL or SQL Server.
• Experience building Databricks dashboards and configuring Genie spaces or agents for business-user analytics.
• Knowledge of schema evolution, MERGE-based incremental loads, liquid clustering or Z-ordering, and performance tuning on large Delta tables.
• Ability to define reusable pipeline patterns, coding standards, and operational practices for a multi-table Lakehouse.
• Databricks Certified Data Engineer Associate, Databricks Certified Data Engineer Professional, or Databricks Certified Platform Administrator certification.

Responsibilities

• Design, build, and operate Databricks Lakehouse solutions in the Azure ecosystem.
• Develop and support Azure Data Factory pipelines that ingest and promote datasets from enterprise applications into Databricks bronze, silver, and gold layers.
• Analyze and troubleshoot pipeline and data-quality problems and resolve data platform issues in collaboration with architects, analysts, and engineering teams.
• Recommend Lakehouse patterns and help determine how source data should land, be refined, and be published for analytics.
• Apply Databricks and Azure best practices to design data models and build reliable ETL/ELT pipelines.
• Design, develop, troubleshoot, and optimize batch and incremental pipelines that move data through bronze, silver, and gold layers.
• Develop and support Databricks analytics, including SQL and Spark datasets, dashboards, Genie agents, and data products for internal teams, external clients, or downstream applications.
• Help design, develop, and test cloud data components.
• Collaborate with clients and developers to ensure requirements are met and solutions are delivered.
• Translate requirements from architects, developers, and mission stakeholders into working Azure Databricks solutions.

Benefits

• Health, life, and disability benefits.
• Financial and retirement benefits.
• Paid leave.
• Professional development.
• Tuition assistance.
• Work-life programs.
• Dependent care.
• Recognition awards for exceptional performance and demonstration of company values.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior