Azure Databricks Engineer

OPTIVEUM sp. z o.o. • Webster, MA

1d • PLN 135 - PLN 160 • Hybrid

About The Position

Optiveum is recruiting and contracting an Azure Databricks Engineer for a project with a leading global investment bank, delivered together with a large international IT services company. The engineer will design and build Azure Databricks big data solutions for batch and real-time analytics, using Delta Lake, PySpark, Kafka streaming, and Lakehouse/Medallion architecture.

This is a full-time B2B role based in Kraków. The work model is hybrid, with three days per week at the client's office required; candidates must live in or near Kraków. The stated rate is 135–160 PLN per hour, depending on experience.

The recruitment process includes an interview with the IT services partner followed by one to three interviews with the bank. Technical tasks are performed live, and AI tools, coding assistants, and code generators are prohibited during interviews. The selected candidate will undergo an external HireRight background check covering identity, right to work, education, and employment history. Application and CV names must match official ID documents exactly. Candidates who participated in a recruitment process for the partner or bank within the previous 12 months are ineligible.

Requirements

• Bachelor's degree or higher in Computer Science, Engineering, or Science.
• 8–10 years of experience, including at least 5 years with Azure cloud-native services.
• Strong hands-on experience with Azure Databricks, Delta Lake, Python/PySpark, and Spark.
• Knowledge of Lakehouse, Data Mesh, Data Virtualisation, dimensional modelling, and data aggregation techniques.
• Experience building large-scale distributed data platforms, such as Spark, Kafka, Flink, or HBase.
• Experience with Hadoop-based implementations, SQL/T-SQL, Unix, and Hive.
• Experience with Azure Data Factory, ADLS Gen2, and Azure SQL Database.
• Good understanding of RDBMS, Data Lake, Data Warehouse, Lakehouse, and Medallion Architecture.
• Experience with at least one workflow orchestration tool, such as Apache Airflow, Luigi, Oozie, AWS Glue, or similar.
• Knowledge of Microsoft Fabric and a basic understanding of data quality dimensions.
• Fluent English.
• Polish or EU citizenship, or permanent residence/work permit valid for at least 12 months.
• Candidates must live in or near Kraków and be able to work from the client's office three days per week.
• Candidates must not have participated in a recruitment process for the IT services partner or bank within the last 12 months.
• Accurate CV dates and employer details are required for the background check; the name on the application and CV must exactly match the official ID document.

Responsibilities

• Design and implement Azure Databricks big data solutions for batch and real-time analytics use cases.
• Build Spark pipelines that populate downstream data marts for BI and generate feeds for Data Science.
• Integrate real-time services that process business events from Kafka and persist them in ADLS Gen2 using Databricks.
• Develop near-real-time stream processing to derive features for ML model inference.
• Optimize Databricks performance, including liquid clustering, vacuum, and Z-ordering.
• Work with DevOps teams to drive operationally excellent infrastructure.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior