Data Engineer

SiloSmashers • Arlington, VA

7d • Onsite

About The Position

SiloSmashers is hiring a Data Engineer to support digital transformation efforts for a government customer. As part of a team, the Data Engineer will architect, develop, optimize, and maintain high-performance data ingestion, transformation, streaming, and analytical pipelines in Databricks-enabled Lakehouse environments. The role involves close collaboration with federal technical teams and personnel across platform, SRE, governance, privacy, and development functions. Regular physical attendance at customer sites is required.

Requirements

• Ability to obtain and maintain a Public Trust clearance.
• Eight or more years of experience in data engineering, ETL/ELT development, or distributed data processing within cloud environments or platforms; at least two of those years must be within the last five years.
• Bachelor’s degree in computer science, information systems, engineering, or a related field, or at least four years of equivalent experience within the last six years in lieu of a degree.
• Databricks Data Engineer Associate Certification.
• At least one foundational cloud certification, such as AWS Cloud Practitioner, AWS Developer Associate, Azure Fundamentals, or equivalent.
• Foundational experience designing and implementing Databricks Lakehouse solutions, including Delta Lake table design, schema evolution, and familiarity with bronze/silver/gold medallion architecture.
• Hands-on experience with Databricks Workflows/Jobs, Auto Loader, COPY INTO, Structured Streaming, and introductory Databricks SQL Warehouse usage.
• Understanding of common performance considerations, including cluster sizing, partitioning, and basic caching strategies.
• Practical experience building batch or streaming pipelines using Python and PySpark, along with cloud-native services in AWS or Microsoft Azure.
• Experience implementing secure cloud data integrations or platform configurations, such as external locations, storage credentials, secret scopes, or identity federation basics.
• Experience with Git-based workflows and CI/CD tooling, such as GitHub Actions or Azure DevOps.
• Regular physical attendance at customer sites is required.

Nice To Haves

• Familiarity with Databricks Repos, Asset Bundles, or basic Terraform/infrastructure-as-code concepts.
• Introductory experience or interest in AI/ML-enabled engineering, such as Databricks Model Serving, Genie, Agent Bricks, or equivalent cloud-native ML platforms.
• Databricks Lakehouse Architect, Generative AI Associate/Professional, or related ML/AI certifications.

Responsibilities

• Architect, develop, optimize, and maintain data ingestion, transformation, streaming, and analytical pipelines in Databricks-enabled Lakehouse environments.
• Collaborate with federal technical teams and platform, SRE, governance, privacy, and development personnel.
• Build batch or streaming data pipelines using Python and PySpark with cloud-native services.

What This Job Offers

• Job Type: Full-time