Databricks Data Engineer, Senior

Booz Allen Hamilton•Bethesda, MD
•$77,600 - $176,000

About The Position

As a senior Databricks Data Engineer, you will design, build, and maintain robust data pipelines in Databricks. You will create and optimize end-to-end data flows, manage ETL processes, design data models, ensure data security and compliance, collaborate with cross-functional teams, and improve performance and scalability. Experience with Databricks and AWS will support your role as a technical team leader, bridging raw data and actionable insights to enable efficient data processing and analytics. You will contribute to and mentor a multidisciplinary team of analysts, data engineers, developers, and data consumers in a fast-paced, agile environment. You will use analytical exploration and data examination skills while overseeing the assessment, design, building, and maintenance of scalable platforms for clients. The position requires eligibility to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements. Applicants selected will be subject to a government investigation and may need to meet U.S. government client eligibility requirements.

Requirements

  • 8+ years of experience designing, developing, deploying, and managing ETL processes and data pipelines, including comprehensive logging and robust error handling for troubleshooting and issue resolution.
  • 3+ years of experience building, debugging, and optimizing data pipelines using Databricks notebooks, clusters, and job orchestration tools, including Delta Lake, Databricks Delta, and Unity Catalog.
  • 3+ years of experience writing Python and PySpark code for ETL scripts and custom transformations.
  • Experience in data modeling, including selecting dimensional versus relational data models within a Bronze, Silver, or Gold medallion architecture.
  • Experience using AWS services, including AWS Glue, Amazon S3, and Amazon RDS.
  • Knowledge of ETL performance optimization techniques, including partitioning, bucketing, caching, and troubleshooting query performance.
  • Ability to design and implement data models, from defining schemas to optimizing data structures for efficient queries and analytics.
  • Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements.
  • Bachelor’s degree in Computer Science, Computer Engineering, Engineering, Mathematics, Physics, Information Systems, or Information Technology.

Nice To Haves

  • Experience with AWS cloud-native data services such as AWS Lambda or CloudWatch that support serverless data processing architectures.
  • Experience with AWS Glue Data Catalog or Collibra for metadata management, data lineage, and data asset cataloging.
  • Knowledge of data governance, data lineage, and compliance requirements, including GDPR or CCPA, to support data quality, privacy, and security.
  • Ability to explain complex technical concepts to non-technical audiences, including architectural tradeoffs.

Responsibilities

  • Design, build, and maintain robust data pipelines within Databricks.
  • Create and optimize end-to-end data flows and ETL processes.
  • Design data models and data structures that support efficient queries and analytics.
  • Ensure data security and compliance.
  • Collaborate with cross-functional teams and contribute to scalable client platforms.
  • Continuously improve data processing performance and scalability.
  • Provide technical leadership and mentorship to analysts, data engineers, developers, and data consumers.
  • Assess, design, build, and maintain scalable platforms.

Benefits

  • Health, life, disability, financial, and retirement benefits.
  • Paid leave.
  • Professional development.
  • Tuition assistance.
  • Work-life programs.
  • Dependent care.
  • Recognition awards for exceptional performance and demonstration of company values.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service