Data Lakehouse Developer

B3 Consulting Poland • Swoyersville, PA

1d

About The Position

Join an international organization listed on Nasdaq Stockholm, operating in 11 countries. In Poland, teams work with autonomy to choose technologies, processes, and ways of working while maintaining high professional standards. The organization delivers IT projects in Poland and internationally across multiple domains and technologies, supporting consultants’ professional development through responsible project matching, transparent cooperation, and a long-term approach to growth.

The project focuses on building and maintaining a scalable Data Lakehouse architecture using Apache Iceberg, Delta Lake, or Apache Hudi. The role processes data across the full pipeline, from raw lake ingestion to refined presentation layers.

Requirements

• At least 4 years of hands-on production experience in data engineering.
• Strong proficiency in Python, PySpark, Spark-SQL, DataFrames, and Structured Streaming.
• Proven experience with MongoDB, including the Spark Connector and indexing, and Apache Iceberg, including table versioning and upserts/merges.
• Hands-on background in CI/CD automation using Jenkins and containerized deployments on Kubernetes.
• Must live in Poland and/or be eligible to work in Poland.

Responsibilities

• Design and maintain versioned Bronze, Silver, and Gold data layers using Apache Iceberg, Delta Lake, or Apache Hudi, including CI/CD for schema evolution.
• Build PySpark and Spark-SQL pipelines for data reconciliation between MongoDB and source systems, optimizing reads through partitioning and match push-down.
• Optimize Spark jobs using AQE, caching, and broadcast joins to reduce latency and Data Lakehouse processing costs.
• Implement automated data quality checks, SLA/SLO tracking, and distributed monitoring for batch and streaming pipelines using Prometheus, Grafana, and Dynatrace.
• Automate Spark application deployments on Kubernetes using Docker, Spark Operator, and CI/CD pipelines.
• Diagnose data discrepancies, maintain technical runbooks, conduct code reviews, and collaborate in Agile teams using Scrum or Kanban.

Benefits

• Health and sports benefits tailored to individual needs.
• Speed Coaching sessions.
• Sports training, an internal team, and opportunities to take part in competitions.
• Team events and development-focused meetups.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed