Senior Data Engineer
About The Position
Clarity Innovations is hiring a Senior Data Engineer to directly support U.S. Special Operations Command (SOCOM) at MacDill Air Force Base in Florida. The role is embedded within SOCOM headquarters and provides subject matter expertise and technical capability for integrating complex data. It supports the command’s data governance and modernization efforts and helps SOCOM use its data resources more effectively. The Senior Data Engineer applies performance optimization, data pipelining, programming, SQL database design, and API integration skills to make raw data useful across the enterprise. The role involves integrating complex models and distributing large, clustered datasets using advanced software library frameworks. It also requires basic machine-learning knowledge to build efficient, accurate data pipelines for downstream analytics users.
Requirements
- At least 1 year of experience.
- Bachelor’s degree in a STEM field; preference for Computer Science or Software Engineering.
- Verifiable work experience with data structures, database management, distributed computing, and API-driven architectures using SQL and NoSQL engines.
- Proficiency with modeling frameworks such as Universal Modeling Language (UML), Agile Development, and Git Operations.
- Proficiency in one or more programming languages.
- Secret clearance.
- Ability to work onsite 5 days per week.
- Experience working with large financial datasets.
Nice To Haves
- Familiarity with Palantir Foundry.
- Familiarity with DoW.
- Top Secret clearance with SCI eligibility.
Responsibilities
- Develop, maintain, and test infrastructures for generating data and transforming data from structured and unstructured sources.
- Develop complex queries to ensure data accessibility while optimizing NoSQL and/or big data infrastructure performance.
- Create and maintain optimal data pipeline architecture.
- Build and maintain infrastructure supporting the extraction, transformation, and loading (ETL) of data from a wide variety of sources.
- Extract data from relational SQL and NoSQL databases and platform APIs for ingestion and integration.
- Configure and manage data analytics frameworks and pipelines using databases and tools such as NoSQL, SQL, HDInsight, MongoDB, Cassandra, Neo4j, GraphDB, OrientDB, Spark, Hadoop, Kafka, Hive, and Pig.
- Apply distributed-systems concepts and principles, including consistency and availability, liveness and safety, durability, reliability, fault tolerance, and consensus algorithms.
- Administer cloud computing and CI/CD pipelines, including Azure, Google, and Amazon Web Services (AWS).
- Coordinate with product, data, and design stakeholders to address data-related technical issues and support data infrastructure needs.
Benefits
- Performance-based bonuses.
- Medical coverage.
- Dental coverage.
- Vision coverage.
- 401(k) plan with company match.
- Substantial paid time off and holidays.
- Flexible work arrangements where possible.
- Income protection benefits.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
