Databricks Engineer
About The Position
Databricks Engineer The Opportunity: As a Data Engineer on the Databricks platform at Booz Allen, you'll support mission-critical data initiatives for defense, intelligence, and civil clients. You'll help design, build, and maintain scalable data pipelines in modern cloud environments, enabling analytics, AI, ML, and decision advantage at scale. Join a team that accelerates national security outcomes through data engineering excellence. Work with cutting-edge tools like Databricks, Delta Lake, and PySpark to transform raw data into actionable insights for America's toughest challenges. Due to the nature of work performed within this facility, U.S. citizenship is required. Work with us to use data for good. What You'll Work On: Build and optimize ELT or ETL pipelines using Databricks Notebooks, Delta Live Tables (DLT), Workflows, and Unity Catalog. Implement data ingestion, transformation, and quality processes with PySpark, Spark SQL, and Delta Lake best practices such as incremental mergers, ACID transactions, or SCD patterns. Collaborate with data scientists, analysts, and mission stakeholders to understand requirements and deliver reliable, governed data assets. Support data platform operations, monitor pipeline performance, troubleshoot issues, and contribute to cost optimization, including leveraging Photon engine. Contribute to medallion architecture implementations, including bronze, silver, and gold layers, in cloud storage environments such as Azure ADLS, or AWS S3. Document pipelines, data models, and processes to ensure maintainability and knowledge transfer. Learn and apply emerging Databricks capabilities to advance client missions. Join us. The world can’t wait.
Requirements
- 2+ years of experience in data engineering, software development, or analytics engineering
- Experience writing SQL queries for data analysis and transformation
- Experience with Python programming such as scripting and data manipulation
- Knowledge of distributed data processing concepts such as Spark fundamentals, including partitioning, caching, and shuffle
- Ability to work in a team environment and communicate technical concepts to non-technical stakeholders
- Ability to obtain and maintain a Public Trust or Suitability/Fitness determination based on client requirements
- Bachelor’s degree in Computer Science, Engineering, Information Systems, or Mathematics
Nice To Haves
- Experience with Databricks, including notebooks, Delta Lake, Workflows, or DLT in academic, internship, personal project, or professional environments
- Experience with cloud platforms such as Azure, AWS, or GCP and object storage
- Experience with orchestration tools such as Databricks Workflows or Airflow or dbt
- Experience with streaming data processing or real-time pipelines
- Knowledge of Git for version control and collaborative development
- Knowledge of data governance concepts, including Unity Catalog and access controls
- Databricks Certified Data Engineer Associate Certification
Responsibilities
- Build and optimize ELT or ETL pipelines using Databricks Notebooks, Delta Live Tables (DLT), Workflows, and Unity Catalog.
- Implement data ingestion, transformation, and quality processes with PySpark, Spark SQL, and Delta Lake best practices such as incremental mergers, ACID transactions, or SCD patterns.
- Collaborate with data scientists, analysts, and mission stakeholders to understand requirements and deliver reliable, governed data assets.
- Support data platform operations, monitor pipeline performance, troubleshoot issues, and contribute to cost optimization, including leveraging Photon engine.
- Contribute to medallion architecture implementations, including bronze, silver, and gold layers, in cloud storage environments such as Azure ADLS, or AWS S3.
- Document pipelines, data models, and processes to ensure maintainability and knowledge transfer.
- Learn and apply emerging Databricks capabilities to advance client missions.
Benefits
- Our offerings include health, life, disability, financial, and retirement benefits, as well as paid leave, professional development, tuition assistance, work-life programs, and dependent care.
- Our recognition awards program acknowledges employees for exceptional performance and superior demonstration of our values.
- Full-time and part-time employees working at least 20 hours a week on a regular basis are eligible to participate in Booz Allen’s benefit programs.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
