Data Engineer-Sr
About The Position
The Senior Data Engineer will lead the backend data engineering team on the Intelligence Data Science Team (IDST), supporting the J2 Intelligence Enterprise at SOCOM Headquarters at MacDill Air Force Base in Florida. The role leads a small team managing data infrastructure with advanced technologies and methodologies, deploys applications using Docker, and works with AWS cloud environments. The position is onsite five days per week. The engineer designs and maintains data systems and pipelines for mission-critical analytics, builds API endpoints for varied data sources, and contributes to data modeling, documentation, and process improvement. The role also leads complex engineering projects, mentors junior staff, and helps deliver strategic initiatives. The position requires a current DoW Top Secret/SCI clearance and U.S. citizenship.
Requirements
- 10 or more years of related experience; internship or academic project experience is accepted.
- Current DoW Top Secret/SCI clearance.
- U.S. citizenship.
- Bachelor’s degree in Data Engineering, Computer Science, Information Systems, or a related field.
- Proficiency in Python, Bash, and SQL.
- Experience with Postgres or Elasticsearch databases.
- Experience setting up Flask or FastAPI applications, Docker containerization, and Git version control.
- Software testing expertise in Python.
- Understanding of data warehousing concepts and cloud technologies such as AWS, Azure, or GCP.
- Familiarity with Databricks, Spark, or similar distributed data tools; academic or project experience is accepted.
- Ability to manage a small technically focused team.
Nice To Haves
- Strong background in Apache Airflow.
- Familiarity with both Postgres and Elasticsearch.
- Advanced skills in Docker, Kubernetes, and Helm charts.
- Experience using GitLab.
- Experience setting up CI/CD pipelines on GitHub or GitLab.
- Knowledge of AI/ML technologies such as HuggingFace, PyTorch, and Transformers, and the SQLAlchemy library.
- Cloud foundational certifications.
- Experience with programming languages and tools including Java, Kafka, Hive, R, or Storm.
Responsibilities
- Design, implement, and maintain reliable, scalable SQL databases.
- Collaborate with cross-functional teams to translate client requirements into solutions using languages and tools such as Python, Java, Kafka, Hive, R, or Storm.
- Oversee cloud-based database platforms and maintain security controls.
- Develop and maintain ETL/ELT pipelines for mission-critical analytics.
- Assist with integrating, cleaning, and validating structured and unstructured data.
- Contribute to data modeling, documentation, and process improvement initiatives.
- Create Flask or FastAPI endpoints to access commercially available, publicly available, and classified information sources.
- Develop Apache Airflow pipelines to perform ETL operations into Postgres or Elasticsearch using Python, Bash, and SQL.
- Lead complex data engineering projects, mentor junior staff, contribute to strategic initiatives, and apply advanced technical expertise to solve data challenges.
- Lead and manage a small technically focused team delivering value to a globally distributed enterprise.
- Deploy applications with Docker and work in AWS cloud environments.
Benefits
- Medical plan options, some with Health Savings Accounts
- Dental plan options
- Vision plan
- 401(k) plan with company match
- Paid time off, including vacation, sick and personal time
- Paid holidays
- Paid parental leave
- Military, bereavement, and jury duty leave
- Short- and long-term disability benefits
- Life insurance
- Accidental death and dismemberment coverage
- Personal accident, critical illness, and business travel and accident insurance
- Wellness packages
- Full flex work weeks where possible
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
