Integration Engineer (Data Tagging and Availability)

i4DM•Millersville, MD

About The Position

Are you ready to help enable data‑driven operations at the Army’s tactical edge? We are seeking an Integration Engineer with strong expertise in data tagging, metadata management, workflow orchestration, Apache Kafka, and cross‑domain integration. In this role, you’ll build and maintain the integration layer that supports automated and AI‑assisted data workflows—ensuring data is properly tagged, governed, and reliably shared across domains, including disconnected and bandwidth‑constrained environments. Your work will support a data mesh or data fabric architecture where data products are discoverable, secure, and interoperable, helping operators and analysts access trusted information when and where it’s needed.

Requirements

Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or a related field.
5+ years of experience in data engineering, data integration, or software engineering with a focus on data governance, metadata management, and enterprise integration.
Proven experience implementing automated data tagging frameworks at enterprise scale.
Hands-on experience with data catalog tools (Alation, Collibra, DataHub, or Amundsen) including API integration and metadata synchronization.
Understanding of metadata standards (DCAT, W3C PROV, OpenLineage) and semantic web concepts.
Experience with data classification and sensitive data detection (PII, PHI, PCI).
Experience with workflow orchestration tools:
Apache Airflow with custom operators, sensors, and DAG design
Alternatives: Prefect, Dagster, AWS Step Functions, Azure Logic Apps
Experience building governed workflows with approval gates, validation steps, and audit trails.
Familiarity with event-driven workflows triggered by Kafka messages or metadata changes.
Proven experience designing and operating cross-domain data integration architectures in large enterprises.
Understanding of data mesh principles and data product orientation with tagging as a core component.
Production experience with Apache Kafka, including:
Kafka cluster administration
Kafka Streams or ksqlDB
Experience embedding metadata and tags within Kafka messages or schema annotations.
Deep experience with enterprise data modeling across multiple domains.
Proficiency with data modeling tools (ERwin, ER/Studio, SAP PowerDesigner).
Experience embedding business tags, classifications, and governance attributes into physical and logical data models.
Advanced proficiency for custom tagging scripts, workflow automation, and Kafka integration.
Experience with Kafka client libraries and stream processing applications.
Expert-level for metadata querying, validation, and lineage extraction.
Deep experience with AWS (MSK, ECS, Lambda, S3, Glue, IAM) or Azure (Event Hubs, Data Factory, Synapse, Purview).
Docker, Kubernetes, Helm for deploying workflow and streaming applications.
Git and CI/CD pipelines (GitHub Actions, GitLab CI, Jenkins).
Infrastructure as Code: Terraform, AWS CloudFormation, or Azure Resource Manager.

Responsibilities

Design and implement automated data tagging frameworks that attach business, technical, and operational metadata to data assets at ingestion time.
Integrate with data catalogs to programmatically populate and maintain metadata, including:
Business glossaries and term definitions
Data classification (PII, sensitive, confidential)
Domain ownership and stewardship
Data quality scores and lineage
Build pipelines that extract metadata from source systems (databases, Kafka schemas, file formats) and synchronize with enterprise metadata repositories.
Implement tagging policies that propagate across domains, ensuring data assets are consistently labeled for discoverability, access control, and retention.
Design and implement governed data workflows that enforce approval gates, validation checks, and compliance requirements before data is published to consumers.
Build workflow automation using tools like Apache Airflow, Prefect, Dagster, or cloud-native workflow services (AWS Step Functions, Azure Logic Apps).
Integrate workflow engines with data catalogs and tagging systems to trigger actions based on metadata changes (e.g., when a dataset is tagged as "sensitive," automatically apply encryption and restrict access).
Implement SLA monitoring and alerting for workflow completion, data freshness, and compliance checks.
Design and implement integration patterns that enable secure, governed data flow across multiple domains.
Implement cross-domain service architectures using APIs, event streaming, and data virtualization.
Define and manage data contracts between domains, specifying schemas, SLAs, quality requirements, and tagging expectations.
Architect, deploy, and manage Apache Kafka clusters across multiple domains and environments (on-premise, cloud, hybrid).
Implement streaming workflows where Kafka messages trigger governed workflows (e.g., new data arrival initiates validation and tagging pipeline).
Design canonical data models that serve as the standard for cross-domain data exchange, embedding tags and metadata into the model structure.
Collaborate with domain experts to define business terms, hierarchies, and metrics that are consistently tagged and governed across domains.
Implement tag-based access control (TBAC) where data access policies are enforced based on tags applied to datasets.
Ensure compliance with regulatory requirements (GDPR, CCPA, SOX) through automated tagging of sensitive data and workflow-enforced retention/deletion policies.
Build data lineage that captures tagging events and workflow approvals, providing end-to-end visibility into how data is governed.
Implement data quality workflows where datasets must pass quality checks before being tagged as "certified" or "trusted."
Set up alerting for workflow failures, tagging inconsistencies, schema drift, and cross-domain connectivity issues.
Document tagging schemas, workflow definitions, and integration patterns for operational handoff.