Databricks Data Engineer, Senior

Booz Allen Hamilton•Bethesda, MD
•$77,600 - $176,000•Hybrid

About The Position

As a senior Databricks Data Engineer, you will design, build, and maintain robust data pipelines within Databricks, contributing independently while providing technical leadership and mentorship. You will create and optimize end-to-end data flows, manage ETL processes, design data models, ensure data security and compliance, collaborate with cross-functional teams, and improve performance and scalability. Your experience with Databricks and AWS will support technical team leadership and help turn raw data into actionable insights for business analytics. You will contribute to and mentor a multidisciplinary team of analysts, data engineers, developers, and data consumers in a fast-paced, agile environment. You will apply analytical exploration and data examination skills while overseeing the assessment, design, development, and maintenance of scalable client platforms.

Requirements

  • At least 8 years of experience designing, developing, deploying, and managing ETL processes and data pipelines, including comprehensive logging and robust error handling.
  • At least 3 years of experience building, debugging, and optimizing pipelines using Databricks notebooks, clusters, and job orchestration tools, including Delta Lake, Databricks Delta, and Unity Catalog.
  • At least 3 years of experience writing Python and PySpark for ETL scripts and custom transformations.
  • Experience with data modeling, including choosing dimensional versus relational models within a Bronze, Silver, or Gold medallion architecture.
  • Experience using AWS services, including AWS Glue, Amazon S3, and Amazon RDS.
  • Knowledge of ETL performance optimization techniques, including partitioning, bucketing, caching, and query-performance troubleshooting.
  • Ability to design and implement data models, from defining schemas to optimizing data structures for queries and analytics.
  • Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.
  • Bachelor’s degree in Computer Science, Computer Engineering, Engineering, Mathematics, Physics, Information Systems, or Information Technology.

Nice To Haves

  • Experience with AWS cloud-native data services such as AWS Lambda or CloudWatch that support serverless data processing.
  • Experience with AWS Glue Data Catalog or Collibra for metadata management, data lineage, and data cataloging.
  • Knowledge of data governance, data lineage, and compliance requirements such as GDPR or CCPA, supporting data quality, privacy, and security.
  • Ability to explain complex technical concepts to non-technical audiences, including architectural tradeoffs.

Responsibilities

  • Design, build, and maintain robust Databricks data pipelines and end-to-end data flows.
  • Develop, manage, and optimize ETL processes, including logging and error handling.
  • Design data models and structures that support efficient queries and analytics.
  • Ensure data security and compliance.
  • Collaborate with cross-functional teams and contribute to scalable data platforms.
  • Improve data processing performance and scalability.
  • Provide technical leadership and mentorship to team members.

Benefits

  • Health, life, disability, financial, and retirement benefits.
  • Paid leave.
  • Professional development.
  • Tuition assistance.
  • Work-life programs.
  • Dependent care.
  • Recognition awards for exceptional performance and demonstration of company values.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service