Senior Data Engineer, Manager

JS Perkins Consulting•Falls Church, VA

2d•Hybrid

About The Position

J.S. Perkins Consulting is seeking a Senior Data Engineer to support a mission-critical Defense Health data modernization initiative. The program is building an enterprise data orchestration layer across a large, federated defense health data landscape. The work connects clinical, operational, financial, and readiness source systems to a modern enterprise data platform, then curates that raw data into governed, well-documented silver and gold data products that mission users can trust and reuse. The model is federated: data stays with its authoritative owners, and the orchestration layer makes it discoverable, interoperable, and consumable through catalogs, APIs, and a data product marketplace. In this role you will serve as a senior technical contributor, delivering production-ready data capabilities while establishing reusable patterns for ingestion, integration, and delivery. You will take on the harder integration problems, help shape how ingestion and curation are done consistently across pipeline and product teams, and support other engineers working through unfamiliar systems. This is a senior individual contributor role with real technical depth, not a management position.

Requirements

8 or more years of data engineering, software engineering, or closely related technical experience
Extensive hands-on experience designing, building, and operating production data pipelines and data products at scale
Expert-level SQL. Able to write, optimize, and debug complex queries for large-scale analysis and transformation, including complex joins, window functions, and common table expressions
Advanced proficiency with Python
Strong experience with Apache Spark and distributed data processing across clusters
Deep familiarity with the Databricks platform, including workspace management, notebook collaboration, and Delta Lake optimization
Experience designing and maintaining ETL and ELT processes for complex, large-scale datasets
Experience integrating data across disparate systems and consuming or developing API-based data integrations
Strong understanding of data modeling, data architecture, data quality, and data governance principles, including establishing data quality rules and anomaly detection
Experience troubleshooting and optimizing complex production pipelines for performance, reliability, and scalability
Experience with Git-based development workflows, code review, and modern software engineering practices
Comfort working in terminal and command-line environments
Demonstrated experience providing technical guidance, mentoring engineers, and influencing engineering practice
Ability to proactively engage subject matter experts, decode complex business logic, and set technical data requirements from it
Strong client and stakeholder communication skills, with the ability to translate technical concepts and recommendations for both technical and non-technical audiences
Ability to independently navigate ambiguity, identify technical risk, and drive complex engineering challenges to resolution across multiple concurrent workstreams
Experience spotting security, privacy, and compliance issues and working them with the appropriate stakeholders
Bachelor's degree in Computer Science, Engineering, Data Science, or a related technical field
U.S. citizenship, and the ability to obtain and maintain a DoW Secret clearance

Nice To Haves

Active DoW Secret clearance or higher at time of application
Master's degree in Computer Science, Engineering, Data Science, or a related technical field
GitLab experience, including code versioning, peer review, and managing CI/CD deployment pipelines
AWS or other cloud platform experience
Experience with Advana, the War Data Platform, Palantir Foundry, or similar DoW data environments
Experience with very large datasets, including datasets containing billions of records
Experience with defense health, federal health, financial, or other regulated and sensitive data, including PHI and PII handling under HIPAA
Familiarity with health data standards such as HL7 or FHIR, ICD-10, or CPT
Experience standing up or operating a data mesh, data product, or federated data governance model
Experience with data catalog, metadata, and lineage tooling and enterprise data product registration
Experience using AI-assisted development tools to accelerate engineering, testing, documentation, and debugging

Responsibilities

Design, build, and operate scalable, production-ready data pipelines and data products using Apache Spark with Python and SQL in a Databricks environment
Lead the integration and transformation of complex data from diverse defense and federal health systems into reliable, reusable silver and gold data products
Design scalable approaches for data ingestion, integration, and exchange across a federated data landscape, including API-based integrations and services
Establish and promote reusable data engineering patterns, standards, and best practices that improve consistency, scalability, and maintainability across data products
Provide technical guidance on data architecture, pipeline design, data modeling, and integration approaches
Define and implement data validation, quality, and governance practices, including data quality rule frameworks and anomaly detection, that make data products trustworthy at every layer
Optimize Delta Lake and Databricks performance for large-scale, high-volume workloads
Troubleshoot complex technical and data integration challenges, identify root causes, and drive durable solutions rather than repeat fixes
Partner with subject matter experts and functional owners to decode complex business logic and convert domain knowledge into defensible technical data requirements
Ensure data products carry the metadata, lineage, and documentation required for enterprise catalog registration and customer data governance expectations
Provide technical guidance and mentorship to other engineers, helping the team navigate complex or unfamiliar technical challenges
Proactively identify opportunities to improve engineering tooling, process, and patterns, and help drive their adoption
Collaborate with engineers, architects, analysts, and customer stakeholders to translate complex data needs into scalable technical solutions