Software Data Engineer

Atlas Technologies•North Charleston, SC

About The Position

Atlas Technologies, Inc., a rapidly expanding company supporting the U.S. Military and other Government entities, is seeking a fast-learning and self-motivated Software Data Engineer to design, build, and maintain the systems and infrastructure that collect, store, and transform data for the Department of Defense (DoD) and the Naval Information Warfare Center (NIWC). This effort is primarily centered around Data Engineering support for PMW160 Tactical Networks.

Requirements

U.S. Citizenship
Possession of and the ability to maintain a Department of Defense (DoD) Secret Level Security Clearance.
3-5 years' experience supporting data engineering efforts and initiatives
Experience with containerization technologies (Docker, Kubernetes) is highly valued
Working knowledge of Agile methodology and DevOps practices
Experience with open-source, distributed event streaming platform used to collect, process, store, and analyze data feeds (Apache Kafka)
Experience with data orchestration tools (Apache Nifi)
Experience with open-source monitoring and observability architectures to include Grafana LGTM+ Stack and Elastic (ELK) Stack
Experience with high-performance, open-source networking infrastructure tools (Nginx, Apache APISIX)
Experience with open-source, distributed event streaming platform used to collect, process, store, and analyze data feeds (Apache Kafka, Debezium)
Experience with data orchestration tools (Apache Nifi)
Experience with open-source data storage and caching services (Postgres, Redis, MinIO)
Strong documentation and communication skills to work effectively across technical teams
Collaborative mindset with ability to work in fast-paced environments
Self-motivated with strong problem-solving skills and attention to detail
Ability to manage multiple projects and priorities simultaneously
Bachelor's degree preferred or equivalent experience

Nice To Haves

Bachelor's degree in Computer Science, Engineering, or related field
5+ years' experience in data engineering roles
Experience with CI/CD tools (Jenkins, GitLab CI, Azure DevOps, GitHub Actions)
Proficiency with version control systems and collaborative development tools
Experience with configuration management and automation platforms (Ansible, Chef, Puppet)
Programming/scripting experience (Python, PowerShell, Bash, Go)
Experience with data engineering design and visualization tools (Swagger)
Knowledge of microservices architecture and serverless computing
Experience with helm charts and deploying managing and scaling applications

Responsibilities

Support designing and building ETL (Extract, Transform, Load) processes and pipelines to move data from disparate sources into centralized repositories
Develop automated deployment pipelines and Infrastructure-as-Code templates using scripts and orchestration tools to automate repetitive data tasks, reducing manual intervention and error
Design and optimize data architectures to improve performance, cost-efficiency, security, and scalability
Design and implement large-scale processing systems and database architecture solutions (e.g., Data Lakes, Data Warehouses)
Provide recommendations for data engineering best practices and process improvements
Contribute to team objectives and support data engineering initiatives
Prepare technical documentation and present architecture proposals to engineering teams
Partner with other engineering groups to understand their data requirements and with leadership to align technical infrastructure with mission objectives