Data Operations Technical Lead

Agile Defense • Tampa, FL

2d • Onsite

About The Position

Design, build, and maintain highly scalable, reliable, and efficient data pipelines for extracting, transforming, and loading (ETL/ELT) data from various sources into data warehouses, data lakes, and other storage systems. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and ensure the efficient flow of data across various platforms and systems. Develop, deploy, and maintain real-time data streaming solutions to process and deliver data with low latency (using tools like Apache Kafka, Flink, or Spark Streaming). Optimize and automate data workflows, ensuring that data pipelines are efficient, reliable, and capable of handling increasing data volumes. Oversee the development of data models and schema designs to ensure accurate, accessible, and high-performance data storage for analytics and reporting. Ensure data quality, integrity, and consistency by implementing data validation, monitoring, and error-handling mechanisms. Monitor and troubleshoot performance bottlenecks in data systems, resolving issues with data processing and data access. Stay current with emerging trends and best practices in data engineering, recommending new tools, technologies, and methods to enhance existing infrastructure. Mentor and guide junior data engineers, promoting best practices in data engineering and fostering a collaborative, high-performance team environment.

Requirements

• Have active active TS/SCI clearance
• Minimum of eight years’ experience managing digital projects in support of SOF Enteprise customers
• Bachelor's degree in Computer Science, Information Systems, Data Engineering, or a related field (or equivalent relevant work experience)
• 5+ years of experience in data engineering or software engineering, with demonstrated experience in designing and managing complex data pipelines.

Nice To Haves

• Three years experience managing SOF agile development processes
• Three years experience managing SOF Data Operations processes and procedures
• Three years experience managing SOF digital support request requirements, translating them from customer requests to assigning customer and digital support teams
• Masters degree preferred

Responsibilities

• Design, build, and maintain highly scalable, reliable, and efficient data pipelines for extracting, transforming, and loading (ETL/ELT) data from various sources into data warehouses, data lakes, and other storage systems.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and ensure the efficient flow of data across various platforms and systems.
• Develop, deploy, and maintain real-time data streaming solutions to process and deliver data with low latency (using tools like Apache Kafka, Flink, or Spark Streaming).
• Optimize and automate data workflows, ensuring that data pipelines are efficient, reliable, and capable of handling increasing data volumes.
• Oversee the development of data models and schema designs to ensure accurate, accessible, and high-performance data storage for analytics and reporting.
• Ensure data quality, integrity, and consistency by implementing data validation, monitoring, and error-handling mechanisms.
• Monitor and troubleshoot performance bottlenecks in data systems, resolving issues with data processing and data access.
• Stay current with emerging trends and best practices in data engineering, recommending new tools, technologies, and methods to enhance existing infrastructure.
• Mentor and guide junior data engineers, promoting best practices in data engineering and fostering a collaborative, high-performance team environment.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior