Big Data Engineer

SAIC•Camp Smith, HI

1d•Onsite

About The Position

We are seeking a Big Data Engineer to design, build, and maintain real-time and near-real-time data ingestion pipelines supporting a mission-critical system and moving data across security domains using cross-domain solutions (CDS) or guards. The role is responsible for reliably flowing streaming data from diverse sources into the data platform while ensuring data quality, observability, and scalability. The engineer will partner with data engineers, platform engineers, analytics teams, and security and governance personnel to deliver trustworthy, low-latency data for operational decisions while meeting cross-domain and data-classification requirements. This is an on-site position in Honolulu, Hawaii.

Requirements

U.S. citizenship and an active TS/SCI clearance.
A Bachelor of Science in Computer Science, Mathematics, Electrical Engineering, Physics, Information Systems, Information Technology, or a related field.
At least 3 years of experience in data engineering, data operations, or DevOps roles supporting production data pipelines.
Proficiency in Python, Bash, or similar scripting languages commonly used in data science or analytics applications.
Working knowledge of the Linux (RedHat) command line and familiarity with Windows environments.
Solid understanding of data engineering fundamentals, including data pipelines, streaming architectures, ETL/ELT concepts, and data-quality principles.
Familiarity with parsing and transforming JSON, XML, CSV, and Excel data.
Typical education and experience: Bachelor's degree and 9 or more years of experience; master's degree and 7 or more years; or PhD or JD and 4 or more years.

Nice To Haves

Experience with streaming or messaging platforms and tools such as Kafka, JMS, and Apache Flink or Spark Streaming.
Experience with monitoring and observability tools such as Grafana, Prometheus, and Elasticsearch.
Experience with data platforms such as Snowflake.
Familiarity with cross-domain solutions or guards, including data diode concepts, transfer validation, and content filtering.
Experience enforcing encryption, authentication and authorization, and data-classification policies in partnership with security and governance teams.
Experience with version-control tools such as Git and Agile/Scrum practices.
Prior experience in a government, defense, or intelligence community environment.

Responsibilities

Help deliver continual data feeds to users and monitor data quality and overall ingestion health.
Design, build, and maintain resilient real-time and near-real-time ingestion pipelines using Apache NiFi and REST APIs, including backpressure, prioritization, retries, and error handling.
Configure and monitor data flows across security domains using cross-domain guards or solutions, ensuring data integrity and compliance with transfer policies.
Parse, transform, validate, and route structured and unstructured data in Excel, CSV, JSON, and XML formats.
Use data manipulation and visualization tools such as Grafana and Prometheus to communicate pipeline status, data quality, and historical trends.
Collaborate with platform, software, and data engineers to configure and continuously improve ingestion pipeline reliability.
Develop and maintain scripts or software to monitor real-time feeds and alert on data timeliness, volume, lineage, and distribution issues.
Use historical pipeline data to identify and implement improvements to ingestion reliability and performance.
Partner with security and governance teams to enforce encryption, authentication, authorization, and data-classification requirements across pipelines and cross-domain transfers.
Write and maintain Python, Bash, or similar scripts to automate data processing, validation, and monitoring.
Document data flows, system configurations, and standard operating procedures.