Data Engineer - Databricks

Steampunk•McLean, VA

2d•$125,000 - $160,000

About The Position

Steampunk is seeking a seasoned Data Engineer to work with its team and clients to develop enterprise-grade data platforms, services, and pipelines in Databricks. The role calls for a technologist with strong communication and customer-service skills and a passion for data and problem solving. The position contributes to Steampunk’s AI & Data Exploitation Practice. The Data Engineer will lead and architect data migrations using Databricks, emphasizing performance, reliability, and scalability. They will assess existing ETL jobs, workflows, data marts, BI tools, and reports; answer technical inquiries about customization, integration, enterprise architecture, and data-product functionality; and support an Agile software development lifecycle. The work involves cloud database, data warehouse, and data mart solutions, preferably on AWS or alternatively Azure or GCP. The projected annual compensation range is $125,000 to $160,000. Steampunk states that compensation depends on factors including geographic location, contractual requirements, education, knowledge, skills, competencies, and experience. Applicants are expected to be on camera during interviews and assessments; the company may take a picture to verify identity and prevent fraud. Steampunk is an equal opportunity employer and participates in E-Verify.

Requirements

Ability to hold a position of public trust with the US government.
2–4 years of industry experience coding commercial software.
2–4 years of direct Data Engineering experience.
Experience with big-data tools such as Databricks, Apache Spark, and Delta Lake.
Experience with relational SQL, preferably T-SQL; alternatively pgSQL or MySQL.
Experience with data pipeline and workflow management tools such as Databricks Workflows, Airflow, or Step Functions.
Experience with AWS cloud services, including Databricks on AWS, S3, EC2, and RDS, or Azure equivalents.
Experience with object-oriented or object-function scripting languages such as PySpark/Python, Java, C++, or Scala.
Experience with Data Lakehouse architecture and Delta Lake or Apache Iceberg.
Advanced working SQL knowledge, including relational databases, query authoring, and optimization, and familiarity with a variety of databases.
Experience manipulating, processing, and extracting value from large, disconnected datasets.
Experience manipulating structured and unstructured data.
Experience architecting transactional and warehouse data systems.
Experience with the SDLC, CI/CD, and operating in development, test, and production environments.
Commitment to data governance.
Experience working in an Agile environment.
Excellent communication and customer-service skills and a passion for data and problem solving.

Nice To Haves

Experience with cloud database, data warehouse, or data mart solutions, preferably AWS; Azure or GCP are alternatives.
Experience with data cataloging tools such as Informatica EDC, Unity Catalog, Collibra, Alation, Purview, or DataZone.
Experience supporting teams that build web-based interfaces, dashboards, reports, analytics, or machine-learning models.

Responsibilities

Lead and architect data migrations using Databricks, focusing on performance, reliability, and scalability.
Assess and understand ETL jobs, workflows, data marts, BI tools, and reports.
Address technical inquiries concerning customization, integration, enterprise architecture, and data-product features and functionality.
Develop enterprise-grade data platforms, services, and pipelines in Databricks with the team and clients.
Support an Agile software development lifecycle.
Inspect existing data pipelines, understand their purpose and functionality, and re-implement them efficiently in Databricks.
Support project teams of developers and data scientists building web-based interfaces, dashboards, reports, analytics, and machine-learning models.
Contribute to the growth of Steampunk’s AI & Data Exploitation Practice.

Stand Out From the Crowd

Upload your resume and get instant feedback on how well it matches this job.

Upload and Match Resume