Data Engineer, Mid

Booz Allen Hamilton•Washington, DC

1d•$62,000 - $141,000

About The Position

Help modernize how a U.S. Army program captures, preserves, and reuses data generated during high-value system test events. Design and implement dependable data storage and processing capabilities that replace manual file transfers and fragmented local copies with secure, organized, searchable repositories. Ensure valuable datasets remain discoverable, accurate, and useful as the deployed software system and its configurations evolve. Collaborate with software developers, systems engineers, test personnel, and AI/ML practitioners to automate data ingestion, quality validation, relevance-based curation, indexing, version tracking, and backup and recovery. Preserve source provenance and test-event and software-version context so trusted datasets can support performance analysis, government reporting, partner-facing metrics, and model training. Build a foundation for future automated dashboards and controlled data access by authorized stakeholders.

Requirements

At least 4 years of experience in data engineering, database development, or data-platform work supporting software or test environments.
Experience with Python, SQL, data transformation, complex queries, Git source control, automated testing, and peer review.
Experience deploying, configuring, securing, monitoring, tuning, backing up, and troubleshooting locally hosted relational or NoSQL databases.
Experience with database schema and data modeling, indexing, query optimization, data integrity, and controlled access.
Experience building and operating scheduled batch or near-real-time ETL/ELT pipelines that ingest data from files, APIs, applications, and databases.
Experience implementing pipeline validation, relevance-based filtering, logging, alerts, automated retries, and failure recovery.
Experience managing retention and reuse through test-event and source metadata, dataset provenance, software-version traceability, relevance-based curation, automated backups, and recovery testing.
Ability to translate developer, tester, and stakeholder needs into supportable data solutions.
Secret clearance.
High school diploma or GED.

Nice To Haves

Experience supporting U.S. Army or other DoD software development, testing, evaluation, or sustainment programs, particularly with test-event datasets, telemetry, application logs, system performance metrics, or technical reporting.
Experience with Redis architectures, including data structures, indexing, and persistence, together with complementary technologies such as PostgreSQL, object storage, or on-premises data platforms for durable, high-performance retrieval.
Experience preparing reproducible AI/ML training, evaluation, or inference datasets through profiling, labeling, normalization, deduplication, feature preparation, and version tracking.
Experience supporting Agile or Scrum software teams with CI/CD and repeatable deployments, including workflow orchestration tools such as Apache Airflow, Dagster, or Prefect and planning or release tools such as GitLab, Azure DevOps, or Jira.
Experience implementing data catalogs, metadata management, data lineage, schema-evolution controls, or dataset versioning using methods and tools such as DVC, lakeFS, data contracts, or comparable governance practices.
Experience building automated performance dashboards and traceable reports with Power BI, Grafana, Tableau, or Python visualization tools, including drill-down links or references to source datasets.
Experience working in Linux-based, on-premises, restricted, or disconnected environments using Docker or other containers, secure networking, role-based access, audit logging, or controlled remote or API access.
Experience with event-driven or distributed data processing such as Kafka or Spark.

Responsibilities

Design and implement dependable data storage and processing capabilities for data generated during system test events.
Replace manual file transfers and fragmented local copies with secure, organized, searchable repositories.
Automate data ingestion, quality validation, relevance-based curation, indexing, version tracking, backup, and recovery.
Preserve source provenance and test-event and software-version context for datasets.
Support performance analysis, government reporting, partner-facing metrics, and model training with trusted datasets.
Build a foundation for future automated dashboards and controlled data access by authorized stakeholders.
Deploy, configure, secure, monitor, tune, back up, and troubleshoot locally hosted relational or NoSQL databases.
Build and operate scheduled batch or near-real-time ETL/ELT pipelines, including validation, logging, alerts, automated retries, and failure recovery.
Manage data retention and reuse through metadata, provenance, software-version traceability, curation, backups, and recovery testing.
Translate developer, tester, and stakeholder needs into supportable data solutions.