Data Engineer

SiloSmashers • Arlington, VA

11d

About The Position

SiloSmashers, Inc. is hiring a Data Engineer to support digital transformation efforts for a government customer. The role is part of a team responsible for architecting, developing, optimizing, and maintaining high-performance data ingestion, transformation, streaming, and analytical pipelines within Databricks-enabled Lakehouse environments. The Data Engineer will collaborate closely with federal technical teams and personnel across platform, SRE, governance, privacy, and development functions. Regular physical attendance at customer sites is required.

Requirements

• At least eight years of experience in data engineering, ETL/ELT development, or distributed data processing within cloud environments or platforms; at least two years of that experience must be within the last five years.
• Bachelor's degree in computer science, information systems, engineering, or a related field, or at least four years of equivalent experience within the last six years in lieu of a degree.
• Databricks Data Engineer Associate Certification.
• At least one foundational cloud certification, such as AWS Cloud Practitioner, AWS Developer Associate, Azure Fundamentals, or equivalent.
• Ability to obtain and maintain a Public Trust clearance.
• Foundational experience designing and implementing Databricks Lakehouse solutions, including Delta Lake table design, schema evolution, and familiarity with bronze/silver/gold medallion architecture.
• Hands-on experience with Databricks Workflows/Jobs, Auto Loader, COPY INTO, Structured Streaming, and introductory Databricks SQL Warehouse usage; understanding of performance considerations such as cluster sizing, partitioning, and basic caching strategies.
• Practical experience building batch or streaming pipelines with Python and PySpark and cloud-native services in AWS or Microsoft Azure.
• Experience implementing secure cloud data integrations or platform configurations, such as external locations, storage credentials, secret scopes, or identity federation basics.
• Experience with Git-based workflows and CI/CD tooling, such as GitHub Actions or Azure DevOps.
• Regular physical attendance at customer sites.

Nice To Haves

• Familiarity with Databricks Repos, Asset Bundles, or basic Terraform/infrastructure-as-code concepts.
• Introductory experience or interest in AI/ML-enabled engineering, such as Databricks Model Serving, Genie, Agent Bricks, or equivalent cloud-native ML platforms.
• Databricks Lakehouse Architect, Generative AI Associate/Professional, or related ML/AI certifications.

Responsibilities

• Architect, develop, optimize, and maintain high-performance data ingestion, transformation, streaming, and analytical pipelines in Databricks-enabled Lakehouse environments.
• Collaborate with federal technical teams and platform, SRE, governance, privacy, and development personnel.
• Design and implement Databricks Lakehouse solutions, including Delta Lake tables and medallion architecture.
• Build batch or streaming pipelines using Python, PySpark, and cloud-native services.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior