Databricks Data Engineer and Scientist

Booz Allen Hamilton • Bethesda, MD

1d • $77,600 - $176,000

About The Position

As a Databricks Data Engineer and Scientist, you will design, build, and maintain robust data pipelines for a critical enterprise Lakehouse, applying data science, AI, and analytics skills. You will develop and optimize end-to-end data flows, manage ETL processes, design data models, help ensure data security and compliance, collaborate with cross-functional teams, and improve performance and scalability. You will use experience with Databricks and AWS, along with SQL, Python, and AI-assisted coding skills, to analyze pipelines, support data modeling, and turn raw data into actionable business insights.

Applicants selected will be subject to a government investigation and may need to meet eligibility requirements of the U.S. government client. The position requires the ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.

The projected annualized compensation range is $77,600 to $176,000 USD. Salary depends on factors including location, education, skills, competencies, experience, contract-specific affordability, and organizational requirements. The posting is scheduled to close within 90 days of its posting date.

The role’s work model is not specified. Booz Allen describes its general expectations for remote, hybrid, and onsite roles, including that remote employees may occasionally be required to work at a company or customer facility. The posting also states that employees working virtually are generally expected to have cameras on during meetings. AI tools are prohibited for assisting with interview responses unless permission is explicitly provided.

Requirements

• 8+ years of experience designing, developing, deploying, and managing ETL processes and data pipelines, including comprehensive logging and error handling to troubleshoot and resolve issues.
• 3+ years of experience building, debugging, and optimizing data pipelines using Databricks notebooks, clusters, and job orchestration tools, including Delta Lake and Unity Catalog.
• 3+ years of experience writing Python and PySpark code for ETL scripts and custom transformations.
• Experience using AI-assisted coding tools such as Claude Code or Cursor for data engineering and data modeling tasks.
• Experience with data modeling, including selecting dimensional or relational models within a Bronze, Silver, and Gold medallion architecture.
• Experience using AWS services, including AWS Glue, Amazon S3, and Amazon RDS.
• Knowledge of ETL performance optimization techniques such as partitioning, bucketing, and caching, and experience troubleshooting query performance.
• Ability to analyze and decompose complex legacy SQL, rebuild data pipelines or models, and compare results to validate accuracy and performance.
• Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.
• Bachelor’s degree in Computer Science, Computer Engineering, Engineering, Math, Physics, or Information Systems and Technology.

Nice To Haves

• Experience with AWS cloud-native data services such as AWS Lambda or Amazon CloudWatch that support serverless data processing.
• Experience with metadata and data cataloging tools such as AWS Glue Data Catalog or Collibra to manage data assets and lineage.
• Knowledge of data governance, data lineage, and compliance requirements such as GDPR and CCPA that support data quality, privacy, and security.
• Ability to explain complex technical concepts to non-technical audiences, including architectural tradeoffs.

Responsibilities

• Design, build, and maintain robust data pipelines for an enterprise Lakehouse.
• Develop and optimize end-to-end data flows and manage ETL processes.
• Design data models and support data modeling tasks.
• Help ensure data security and compliance.
• Collaborate with cross-functional teams.
• Improve data pipeline performance and scalability.
• Analyze pipelines and turn raw data into actionable insights across the business.
• Troubleshoot and resolve pipeline issues using comprehensive logging and error handling.
• Rebuild data pipelines or models from complex legacy SQL and compare results to validate accuracy and performance.

Benefits

• Health, life, and disability benefits.
• Financial and retirement benefits.
• Paid leave.
• Professional development.
• Tuition assistance.
• Work-life programs.
• Dependent care.
• Recognition awards program for exceptional performance and demonstration of company values.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior