Senior Data Engineer - Databricks

Dataeconomy • Raleigh, NJ

13h • Onsite

About The Position

DATAECONOMY is a rapidly growing Data & Analytics company with a global presence, recognized for its thought leadership, innovative products, cutting-edge solutions, accelerators, and cost-effective service offerings. We specialize in Cloud, Data Engineering, Data Governance, AI/ML, DevOps, and Blockchain, serving large corporations worldwide. We are strategic partners with AWS, Collibra, Cloudera, Neo4j, DataRobot, Global IDs, Tableau, MuleSoft, and Talend. This role is for a Senior/Lead Data Engineer focusing on Databricks, located in Rutherford, NJ/ Jersey City, NJ, and is a full-time position.

Requirements

• Hands-on experience: Spark, Delta Lake, Workflows, Unity Catalog.
• Strong SQL programming and performance tuning skills.
• Experience with cloud environments (AWS/Azure/GCP).
• Experience with modern data lakehouse concepts and distributed systems.
• Strong understanding of Lakeflow Connect, LSDP/Lakehouse, Medallion Architecture, Data Validations, Genie, and Agent Bricks/RAG use cases.
• Should be able to explain these concepts using real project examples and architecture decisions.
• Knowledge of medallion architecture, DLT and unity catalog within Databricks.
• Strong Python (PySpark) and SQL programming.
• Databricks — Spark, Delta Lake, Workflows, Unity Catalog.
• ETL/ELT pipeline development — Medallion Architecture (Bronze/Silver/Gold).
• Delta Live Tables, Auto-Loader, Structured Streaming.
• Data modeling — dimensional (star/snowflake), normalization/denormalization.
• CI/CD, Git, job orchestration.
• Cloud experience — AWS, Azure, or GCP.
• 7–10+ years in data engineering.
• Knowledge of medallion architecture, DLT and unity catalog within Databricks.

Nice To Haves

• Lakeflow Connect, LSDP/Lakehouse, Genie, Agent Bricks/RAG use cases.
• Data governance, metadata management, Unity Catalog advanced features.
• Airflow, dbt, or similar orchestration tools.
• Data security, compliance, and access models.
• Cost optimization and performance tuning in cloud environments.
• Corporate/enterprise data warehousing background.

Responsibilities

• Build scalable, production-grade ETL/ELT pipelines using Databricks (PySpark, Spark SQL, Delta Live Tables, Workflows).
• Ingest structured, semi-structured, and streaming data into Bronze, Silver, and Gold layers.
• Develop optimized transformations, data quality rules, and reusable framework components.
• Implement best practices for job orchestration, monitoring, alerting, and automation.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed