Senior Data Engineer - TS/SCI

Sunayu•Bethesda, MD

1d•Hybrid

About The Position

Sunayu, LLC is seeking a mission-driven Senior Data Engineer to join our high-impact team supporting the National Digital Exploitation and Open Source Center (NDOC) DOMEX Technology Platform (DTP). In this role, you will help improve data intelligence pipelines serving both Open Source Intelligence (OSINT) operations and document media exploitation for the Department of War (DoW) and the broader Intelligence Community (IC). You’ll play a critical role in developing a future state AI-managed DOMEX Data Discovery Platform (D3P) and DIA-NDOC’s mission. As a key technical leader, you will collaborate with talented engineers, product owners, and government stakeholders to design, develop, and deploy innovative software solutions that make a real difference to national security. You’ll have the opportunity to shape the architecture, drive technical direction, and mentor others, all while working in a collaborative, agile, and growth-oriented environment. You will work closely with the team on the following key tasks As a Data Engineer, you will design, develop, and optimize scalable data management solutions supporting advanced analytics and AI/ML workloads. Working closely with data scientists, software engineers, and infrastructure teams, you will build and maintain Elasticsearch and graph-based data platforms that enable efficient storage, discovery, and analysis of complex hierarchical data. Design and optimize Elasticsearch indices to efficiently manage large-scale hierarchical classification structures, ensuring high-performance search, aggregation, and update operations. Design, implement, and maintain a graph data store supporting hierarchical graph structures optimized for large, dense batch updates and high-throughput analytics. Develop scalable ingestion pipelines that support scalable graph analytics. Design data schemas, indexing strategies, and partitioning approaches to maximize query performance, scalability, and storage efficiency. Collaborate with infrastructure and platform engineering teams to deploy, operate, and optimize Elasticsearch and graph database services in a Kubernetes environment using cloud-native technologies. Partner with Data Scientists and AI/ML engineers to ingest, curate, manage, analyze, and securely expunge datasets throughout the data lifecycle. Develop automated data loading, transformation, validation, and quality assurance pipelines supporting production analytics workflows. Optimize graph and search infrastructure for high availability, resilience, and performance under large-scale analytical workloads. Collaborate with software engineers and system architects to integrate graph and search services into microservice-based applications. Participate in SAFe Agile development activities, including sprint planning, design reviews, architecture discussions, and technical demonstrations. Foster a culture of innovation, collaboration, and professional development within the team. Ensure sound engineering practices, compliance with policies, and delivery of high-quality software. Coordinate with test teams to develop and monitor automated system integration tests. Engage with cross-functional teams to identify and develop high-value integrations with other systems/applications. Provide specific input to the software components of system design to include hardware/software trade-offs, software reuse, use of COTS/GOTS in place of new development, and requirements analysis and synthesis from system level to individual software components.

Requirements

Bachelor’s degree in Computer Science, Data Engineering, or related field and 12–15 years of relevant experience, or Master’s with 10–13 years of experience.
Must possess an Active Top Secret/SCI clearance and ability to obtain and maintain a Polygraph.
Elasticsearch/OpenSearch architecture, index design, mappings, analyzers, and query optimization.
Graph databases such as JanusGraph, Neo4j, TigerGraph, Amazon Neptune, or Memgraph.
Data modeling for hierarchical and highly connected datasets.
Large-scale ETL/ELT pipeline development.
Kubernetes deployment and operations for stateful data services.
Distributed storage systems and high-volume data ingestion.
Python or Java for data engineering and automation.

Nice To Haves

Experience supporting AI/ML and data science workflows.
Apache Kafka or similar streaming technologies.
Experience with Linux, containerization, CI/CD pipelines, and Infrastructure-as-Code.
Graph analytics frameworks and algorithms, including graph traversal, centrality, community detection, similarity analysis, and link prediction.
Experience with Keycloak, and security best practices (authN/Z, secrets management).
Experience building and maintaining microservices in Kubernetes.
Familiar with infrastructure-as-code (CloudFormation, Terraform, Pulumi).

Responsibilities

Design, develop, and optimize scalable data management solutions supporting advanced analytics and AI/ML workloads.
Build and maintain Elasticsearch and graph-based data platforms that enable efficient storage, discovery, and analysis of complex hierarchical data.
Design and optimize Elasticsearch indices to efficiently manage large-scale hierarchical classification structures, ensuring high-performance search, aggregation, and update operations.
Design, implement, and maintain a graph data store supporting hierarchical graph structures optimized for large, dense batch updates and high-throughput analytics.
Develop scalable ingestion pipelines that support scalable graph analytics.
Design data schemas, indexing strategies, and partitioning approaches to maximize query performance, scalability, and storage efficiency.
Collaborate with infrastructure and platform engineering teams to deploy, operate, and optimize Elasticsearch and graph database services in a Kubernetes environment using cloud-native technologies.
Partner with Data Scientists and AI/ML engineers to ingest, curate, manage, analyze, and securely expunge datasets throughout the data lifecycle.
Develop automated data loading, transformation, validation, and quality assurance pipelines supporting production analytics workflows.
Optimize graph and search infrastructure for high availability, resilience, and performance under large-scale analytical workloads.
Collaborate with software engineers and system architects to integrate graph and search services into microservice-based applications.
Participate in SAFe Agile development activities, including sprint planning, design reviews, architecture discussions, and technical demonstrations.
Foster a culture of innovation, collaboration, and professional development within the team.
Ensure sound engineering practices, compliance with policies, and delivery of high-quality software.
Coordinate with test teams to develop and monitor automated system integration tests.
Engage with cross-functional teams to identify and develop high-value integrations with other systems/applications.
Provide specific input to the software components of system design to include hardware/software trade-offs, software reuse, use of COTS/GOTS in place of new development, and requirements analysis and synthesis from system level to individual software components.