Sr. Data Engineering Specialist

Leidos•St. Louis, MO

21h•Onsite

About The Position

The Leidos Digital Modernization Group seeks a Senior Data Engineering Specialist with an extensive Data engineering background to support the Global Management Systems (GMS) Team for the Global Solutions Management – Operations II (GSM-O II) contract. This contract includes the Operations, Sustainment, Maintenance, Repair and Defense of the Defense Information System Network (DISN) within the DOD Information Network (DODIN) in support of the Defense Information Systems Agency (DISA). It also includes support for other key tasks for DISA, including the transformation of DISA’s operational mission through innovation, and support to DISA’s mission partners. The candidate will be responsible for providing technical leadership in data engineering, serving as a customer-facing expert on DISN network topology data to enable advanced data correlation and enrichment, and representing the team in technical discussions with internal and external stakeholders. They will design and implement robust data enrichment pipelines, integrating diverse data sources into Confluent (Kafka) and Elastic platforms, and manage complex, multi-site Kafka and Elastic clusters to support mission-critical operations. The candidate will ensure platform sustainment and reliability, proactively addressing operational challenges, and will lead the automation of the software development lifecycle, including CI/CD pipeline creation, containerization, and automated testing, while driving DevOps best practices. Collaboration within Agile scrum teams, mentoring junior members, and fostering continuous improvement are key aspects of the role. Additionally, the candidate will author and maintain comprehensive technical documentation, ensuring all solutions meet DoD security standards and compliance requirements. As a GMS team member, you will work as part of a fast paced, Agile development and implementation team to architect, design and develop an integrated solution that expands the foundational Integrated Data Architecture platform (Confluent and ELK platform). You will work alongside others in a matrixed organization across the project. Candidate must be within driving distance of Scott AFB. At a minimum, a Secret clearance and Security + certification upon the start of employment.

Requirements

Bachelor’s degree from an accredited college in a related discipline, or equivalent experience/combined education, with 8-12 years of professional experience; or 6-10 years of professional experience with a related master’s degree.
8+ years of experience in software engineering, data engineering, or business/data analysis, with demonstrated leadership in Agile/Scrum teams.
Extensive hands-on software development experience with Python, Java, SQL, and working knowledge of JavaScript and HTML.
Advanced experience with distributed version control systems such as Git and Bitbucket.
Deep expertise in data analytics and visualization tools, including Kibana, Power BI, Tableau, and the ELK stack (Elasticsearch, Logstash, Kibana).
Proven experience designing, developing, and optimizing ETL processes and complex data pipelines, including integration with event streaming platforms such as Kafka.
Strong background in data modeling, unification, and analytics to support data-driven projects and advanced enrichment.
Demonstrated ability to architect and implement application and system integrations, including Kafka and Elastic platform integrations.
Solid understanding of networking and internet protocols, with experience supporting network-centric and data-driven environments.
Experience developing and deploying software on UNIX/Linux command line platforms.
Strong facilitation, stakeholder engagement, and communication skills, with the ability to bridge business and technical perspectives and collaborate with customer representatives, domain experts, systems engineers, and architects.
Experience with Agile project management and collaboration tools such as JIRA and Confluence.
Demonstrated leadership in mentoring junior team members and fostering a culture of continuous improvement.
Active Secret DoD Security clearance prior to start date.
Active Security+ Certification (or other applicable DoD 8570 IAT II certification) prior to start date.

Nice To Haves

Advanced experience with CI/CD techniques, containerized pipelines, and DevOps practices, including architecting and automating end-to-end software delivery processes.
Proven expertise in artificial intelligence and machine learning techniques, with a demonstrated ability to design, build, and integrate AI/ML capabilities into production data platforms.
Deep experience in data integration, storage, and analysis technologies such as Kafka, Elastic, Spark, and NiFi, including architecting scalable, high-performance data solutions.
Extensive experience with Kafka connector integrations, custom connector development, and strong working knowledge of ksqlDB and Kafka Streams for real-time data processing.
Skilled in developing detailed software designs for streaming data applications, particularly using ksqlDB and Kafka Streams.
Demonstrated ability to develop and optimize Kafka system integrations between Elasticsearch/Logstash and other enterprise systems.
Expertise in designing and implementing application deployment pipelines and developing software in containerized environments using Kubernetes and Docker.
Strong familiarity with Kubernetes deployment strategies, Agile methodologies, and related tools for collaborative development.
Experience developing and deploying software in AWS cloud environments, including configuration of cloud infrastructure, networking, and security policies (GovCloud experience preferred).
Comprehensive experience in full software lifecycle automation (design, development, testing, deployment), including production-grade deployments.
Advanced skills in designing and building automated software testing pipelines using tools such as Ansible, Selenium, JMeter, Junit, and others.
Demonstrated experience developing and deploying software in DoD environments (DISA experience preferred), including building applications that meet DoD security standards, updating code for security scans, and implementing security guidelines (e.g., STIGs).
Ability to develop DoD requirements, traceability matrices, and detailed project plans/schedules; author software systems engineering documents and interface documents (IDDs/ICDs).
Extensive experience with Agile methodologies and Atlassian tools, including JIRA and Confluence, for project tracking and collaboration.
Proven ability to work effectively in remote, geographically dispersed teams, demonstrating strong communication and collaboration skills.

Responsibilities

Provide strategic technical leadership in data engineering, serving as the primary customer-facing expert on DISN network topology data to enable advanced data correlation and enrichment.
Represent the team in high-level technical discussions and decision-making with internal and external stakeholders, influencing solution design and architecture.
Architect, design, develop, document, test, and deploy complex software solutions using Python, JavaScript, Java, and SQL, ensuring scalability and maintainability.
Lead the creation of advanced Kibana visualizations and dashboards, leveraging machine learning capabilities and React to deliver actionable insights.
Oversee the design and implementation of robust data enrichment pipelines, integrating diverse data sources into Confluent (Kafka) and Elastic platforms.
Direct the development and management of multi-site Kafka and Elastic clusters, including deployment, configuration, testing, troubleshooting, and maintenance to support mission-critical operations.
Lead Kafka system integrations between Elasticsearch/Logstash and other enterprise systems.
Guide Agile collaboration, leading scrum teams, mentoring junior engineers, and fostering a culture of continuous improvement and innovation.
Communicate and collaborate effectively with geographically distributed teams to ensure alignment and successful completion of project objectives.
Troubleshoot and resolve complex installation, infrastructure, and system issues; proactively report, monitor, and mitigate technical risks.
Author, review, and maintain comprehensive technical documentation, including DoD requirements, interface documents, and security compliance artifacts.
Ensure all solutions adhere to DoD security standards and guidelines, and provide oversight for platform sustainment and reliability, proactively addressing operational challenges.