Principal Data Engineer

Confiz

3d

About The Position

Confiz is seeking a Principal Data Engineer with 5+ years of hands-on experience in the modern cloud data stack. This role is ideal for someone passionate about building robust, scalable data solutions and who thrives in agile, collaborative environments. The core technical stack centers around Databricks / PySpark for large-scale data processing, DBT for transformation, Astronomer / Airflow for orchestration (with Cosmos to run DBT DAGs natively in Airflow), and Snowflake as the cloud data warehouse.

Requirements

• 5+ years of hands-on experience in the modern cloud data stack.
• Databricks & PySpark: Delta Lake, batch and streaming workloads, cluster and cost optimization.
• DBT (Core & Cloud): Advanced data modelling patterns, incremental strategies, custom tests, and CI/CD integration.
• Astronomer / Airflow: DAG authoring, Cosmos for DBT orchestration, and pipeline observability.
• Snowflake: Data modelling, query performance tuning, access control, and cost governance.
• Python and SQL: Strong programming skills with a focus on modular, maintainable, and testable code.
• Cloud platform: Hands-on experience with Azure or AWS.
• Agile & CI/CD: Comfortable working in agile teams with a CI/CD development mindset.
• Bachelor’s in Computer Science or a related discipline.

Responsibilities

• Build and maintain scalable data pipelines using DBT and Astronomer/Airflow for batch and incremental data processing.
• Develop modular DBT transformation layers (staging → intermediate → mart) with robust testing, documentation, and incremental model strategies.
• Orchestrate end-to-end workflows on Astronomer / Airflow, using Cosmos to trigger and manage DBT models as native Airflow DAGs.
• Design and optimize Snowflake schemas, queries, and cost controls for analytical workloads.
• Enforce data quality standards through DBT tests and pipeline observability tooling.
• Write clean, well-tested Python and SQL code and actively participate in code reviews.
• Lead and mentor a team of data engineers; contribute to sprint planning, architectural decisions, and technical standards.

What This Job Offers

• Job Type: Full-time
• Career Level: Principal