Data Engineer - Databricks

Steampunk • McLean, VA

1d • $110,000 - $160,000

About The Position

We are looking for seasoned Data Engineers to work with our team and our clients to develop enterprise-grade data platforms, services, and pipelines. We are looking for more than just a "Data Engineer", but a technologist with excellent communication and customer service skills and a passion for data and problem-solving. You will contribute to the growth of our Data Exploitation Practice!

Requirements

• Ability to hold a position of public trust with the US government.
• Bachelor’s Degree and 6+ years of total experience or equivalent experience and education.
• 6+ years direct experience in Data Engineering with experience in tools such as:
• Big data tools: Hadoop, Spark, Kafka, etc.
• Relational SQL and NoSQL databases, including Postgres and Cassandra.
• Data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
• AWS cloud services: EC2, S3, RDS, Glue, Step Functions, Lamda, EMR, DynamoDB, DocumentDB, Redshift, Aurora, Athena
• Data Platforms: Databricks, Snowflake
• Data streaming systems: Kafka, Storm, Spark-Streaming, etc.
• Languages: Python, R, Scala, Go
• Ability to inspect existing data pipelines, discern their purpose and functionality, and re-implement them efficiently in Databricks.
• Advanced working SQL knowledge and experience working with relational databases
• Advanced working knowledge and of NoSQL databases
• Experience with message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Experience manipulating, processing, and extracting value from large, disconnected datasets.
• Experience manipulating structured and unstructured data for analysis.
• Experience with data modeling tools and processes.
• Experience aggregating and transforming data from multiple datasets to create data products
• Experience working in an Agile environment

Nice To Haves

• Preferably AWS. Alternatively Azure, GCP.

Responsibilities

• Lead and architect data pipelines and ingest patterns to move raw data from data producers to an enterprise data ecosystem, with a focus on performance and reliability
• Assess and understand the ETL jobs, workflows, BI tools, and reports
• Address technical inquiries concerning customization, integration, enterprise architecture and general feature / functionality of data products
• Experience in crafting database / data warehouse solutions in cloud (Preferably AWS. Alternatively Azure, GCP).
• Support an Agile software development lifecycle

Benefits

• As an employee owned company, we focus on investing in our employees to enable them to do the greatest work of their careers – and rewarding them for outstanding contributions to our growth.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior