Senior Data Engineer

Arctiq • New York, NY

6d

About The Position

Arctiq is a leader in professional IT services and managed services across three core Centers of Excellence: Enterprise Security, Modern Infrastructure and Platform Engineering. Renowned for our ability to architect intelligence, we connect, protect, and transform organizations, empowering them to thrive in today's digital landscape. Arctiq builds on decades of industry expertise and a customer-centric ethos to deliver exceptional value to clients across diverse industries. We are looking for a Data Engineer to lead the development of scalable data pipelines within the Databricks ecosystem. You will be responsible for architecting robust ETL/ELT processes using a "configuration-as-code" approach, ensuring our data lakehouse is governed, performant, and production-ready. This is a 3-month contract opportunity at 25 hours per week.

Requirements

• Expertise: Deep mastery of PySpark and advanced SQL .
• Platform: Extensive experience in the Databricks environment (Workflows, Delta Lake).
• Cloud: Familiarity with AWS infrastructure and cloud-native data patterns.

Responsibilities

• Pipeline Architecture: Design and implement declarative data pipelines using Lakeflow and Databricks Asset Bundles (DABs) to ensure seamless CI/CD.
• Data Ingestion: Build efficient, scalable ingestion patterns using AutoLoader and Change Data Capture (CDC) to handle high-volume data streams.
• Governance & Security: Manage metadata, lineage, and access control through Unity Catalog .
• Orchestration: Develop and maintain complex workflows using Databricks Jobs and orchestration tools.
• Infrastructure as Code: (Asset) Utilize Terraform to manage AWS resources (S3, EC2) and Databricks workspaces.

What This Job Offers

• Job Type: Part-time
• Career Level: Mid Level
• Education Level: No Education Listed