Data Engineer, Senior Consultant

JS Perkins Consulting•Falls Church, VA

23h•$130,000 - $140,000•Hybrid

About The Position

J.S. Perkins Consulting is seeking a Data Engineer to support a mission-critical Defense Health data modernization initiative. The program is building an enterprise data orchestration layer across a large, federated defense health data landscape. The work connects clinical, operational, financial, and readiness source systems to a modern enterprise data platform, then curates that raw data into governed, well-documented silver and gold data products that mission users can trust and reuse. The model is federated: data stays with its authoritative owners, and the orchestration layer makes it discoverable, interoperable, and consumable through catalogs, APIs, and a data product marketplace. This is a hands-on build role. You will work directly in Spark and Databricks to land data from clinical and operational source systems, transform it into curated silver and gold datasets, and make those datasets reliable enough that mission users trust them without checking your work. The right candidate is strong in SQL, comfortable in distributed processing, disciplined about data quality, and willing to sit down with a subject matter expert to figure out what a messy source field actually means.

Requirements

3 to 10 years of data engineering, software engineering, or closely related technical experience, with 5 or more years preferred
Expert-level SQL. Able to write, optimize, and debug complex queries for large-scale analysis and transformation, including complex joins, window functions, and common table expressions
Strong hands-on experience with Apache Spark, processing large datasets efficiently across distributed clusters
Hands-on experience with the Databricks platform, including workspace management, notebook collaboration, and Delta Lake optimization
Strong proficiency with Python
Demonstrated experience building, monitoring, and maintaining robust end-to-end ETL and ELT pipelines in production
Solid understanding of data quality and data governance practice, including creating data quality rules and implementing anomaly detection to protect data integrity and consistency
Working knowledge of data modeling and turning raw data into reusable analytics datasets and data products
Experience integrating or consuming data through REST APIs
Experience with Git-based development workflows and modern software engineering practices
Comfort working in terminal and command-line environments
Ability to proactively engage subject matter experts, decode complex business logic, and turn domain knowledge into technical data requirements
Strong communication skills, including the ability to explain technical data concepts to non-technical stakeholders and deliver solutions that meet a stated business need
Demonstrated ability to turn messy data into structured, high-value assets that support informed decision-making
Ability to manage multiple data workstreams at once, navigate ambiguity, and drive assigned work to resolution without close supervision
Bachelor's degree in Computer Science, Engineering, Data Science, or a related technical field
U.S. citizenship, and the ability to obtain and maintain a DoW Secret clearance

Nice To Haves

Active DoW Secret clearance or higher at time of application
GitLab experience, including code versioning, peer review, and managing CI/CD deployment pipelines
AWS or other cloud platform experience
Experience with Advana, the War Data Platform, Palantir Foundry, or similar DoW data environments
Experience with very large datasets, including datasets containing billions of records
Experience with defense health, federal health, financial, or other regulated and sensitive data, including PHI and PII handling under HIPAA
Familiarity with health data standards such as HL7 or FHIR, ICD-10, or CPT
Experience with data catalog, metadata, and lineage tooling
Experience using AI-assisted development tools to accelerate routine engineering, testing, documentation, and debugging

Responsibilities

Design, build, and maintain production data pipelines using Apache Spark with Python and SQL in a Databricks environment
Ingest and integrate data from diverse defense and federal health source systems across clinical, operational, financial, and readiness domains
Transform raw source data into curated silver and gold data products with documented business logic, clear ownership, and consistent definitions
Build and maintain API-based integrations for ingesting and exchanging data across federated systems and platforms
Implement data quality controls, including rule creation, anomaly detection, and validation checks that hold data integrity and consistency across every layer of the architecture
Optimize Delta Lake tables and Databricks workloads for performance, cost, and reliability
Monitor, troubleshoot, and tune scheduled workflows and production pipelines, resolving failures and performance issues at the root cause
Produce and maintain the metadata, lineage, and documentation needed to register data products in the enterprise data catalog and marketplace
Engage subject matter experts directly to decode complex business logic and translate domain knowledge into technical data requirements
Contribute to code reviews, automated testing, version control practices, and reusable engineering patterns that raise consistency across the team
Manage multiple concurrent data workstreams in a fast-paced delivery environment and communicate status, risks, and blockers early