Full-Stack Data Engineer

Prescient Edge • Doral, FL

3d • Onsite

About The Position

Prescient Edge is seeking a Full-Stack Data Engineer to support a Federal Government client. This position is contingent upon contract award. The role involves developing, maintaining, and optimizing an outward-facing data catalog that integrates geospatial and research data layers. Responsibilities include optimizing data integration pipelines, API services, and front-end interfaces for accessing and visualizing geospatial and structured data. The engineer must be proficient in full-stack development, database architecture, and API integration to enable seamless search, filtering, and rendering of structured research products. Proficiency in Data Lakes platforms such as Databricks, Kubernetes, ESRI ArcGIS environments, and cloud-based data engineering is required. Expertise in building scalable, secure web applications, implementing metadata-driven search and retrieval features, and integrating APIs for seamless data interoperability is essential. The role also includes establishing and maintaining metadata governance frameworks, ensuring compliance with DoD data management policies and interoperability with a Data Catalog. Developing automated metadata tagging processes to improve data discoverability, classification, and lineage tracking within the Common Intelligence Picture (CIP) platform is a key responsibility. Implementing and managing data governance workflows, ensuring integration with Open Lineage, Unity Catalog, and GIS metadata standards, and submitting the Metadata & Data Governance Compliance Report are also required. Furthermore, the engineer will develop and maintain the data pipeline infrastructure, ensuring scalable ingestion, transformation, and integration of structured and unstructured datasets within the cloud-based CIP platform. Optimizing data storage strategies, implementing efficient query execution, indexing, and retrieval techniques for high-volume mission datasets, and integrating APIs for secure data sharing across operational environments, ensuring controlled access and compliance with security frameworks are also part of the role. The engineer will submit the Data Pipeline Optimization Report, detailing improvements in ingestion speed, storage efficiency, and API integrations.

Requirements

• Demonstrated experience in developing interactive web-based data portals.
• Demonstrated experience in implementing API driven data exchange.
• Demonstrated experience in integrating geospatial data layers into web applications.
• Experience working with Databricks.
• Experience working with ESRI ArcGIS Feature Services.
• Experience working with Open Lineage.
• Experience working with metadata management solutions.
• Demonstrated experience in developing and deploying full-stack applications that enable data visualization, dashboarding, and analytics within ESRI based GIS platforms, Unity Catalog, and cloud-based databases.
• Demonstrated experience in implementing RESTful and GraphQL APIs for seamless data retrieval and integration between Data Lake, ESRI ArcGIS, and external sources.
• Demonstrated experience in managing CI/CD pipelines.
• Demonstrated experience in containerized deployments in Kubernetes.
• Demonstrated experience in API authentication mechanisms to ensure secure access to research products.
• Demonstrated experience in optimizing database queries and indexing strategies for large-scale geospatial and structured datasets.
• Bachelor’s degree in computer science, Data Engineering, Geospatial Information Systems (GIS), or a related field, or five (5) years of equivalent experience in data engineering, full-stack development, and metadata-driven data cataloging.
• TS/SCI security clearance.

Nice To Haves

• Microsoft Certified: Azure Data Engineer Associate certification.
• AWS Certified Data Analytics – Specialty certification.
• ESRI Web GIS Developer Certification.

Responsibilities

• Develop, maintain, and optimize an outward-facing data catalog that integrates geospatial and research data layers.
• Optimize data integration pipelines, API services, and front-end interfaces for accessing and visualizing geospatial and structured data.
• Enable seamless search, filtering, and rendering of structured research products through full-stack development, database architecture, and API integration.
• Build scalable, secure web applications.
• Implement metadata-driven search and retrieval features.
• Integrate APIs for seamless data interoperability.
• Establish and maintain metadata governance frameworks, ensuring compliance with DoD data management policies and interoperability with a Data Catalog.
• Develop automated metadata tagging processes to improve data discoverability, classification, and lineage tracking within the Common Intelligence Picture (CIP) platform.
• Implement and manage data governance workflows, ensuring integration with Open Lineage, Unity Catalog, and GIS metadata standards.
• Submit the Metadata & Data Governance Compliance Report, detailing lineage tracking, classification policies, and security enhancements.
• Develop and maintain the data pipeline infrastructure, ensuring scalable ingestion, transformation, and integration of structured and unstructured datasets within the cloud-based CIP platform.
• Optimize data storage strategies, implementing efficient query execution, indexing, and retrieval techniques for high-volume mission datasets.
• Integrate APIs for secure data sharing across operational environments, ensuring controlled access and compliance with security frameworks.
• Submit the Data Pipeline Optimization Report, detailing improvements in ingestion speed, storage efficiency, and API integrations.

Benefits

• Competitive salary with performance bonus opportunities.
• Comprehensive healthcare benefits, including medical, vision, dental, and orthodontia coverage.
• A substantial retirement plan with no vesting schedule.
• Career development opportunities, including on-the-job training, tuition reimbursement, and networking.
• A positive work environment where employees are respected, supported, and engaged.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level