Senior Data Engineer (Azure Databricks)

Applaudo Studios

23h

About The Position

Applaudo is seeking a Senior Data Engineer to build and maintain scalable, production-grade data pipelines and transform data from multiple sources into reliable, business-ready datasets. The role uses Azure Databricks, Python, and Apache Spark to integrate data across enterprise platforms, optimize transformations, and support accurate, consistent data throughout its lifecycle. The engineer will contribute to a centralized Data Hub, develop data warehouse structures and dimensional models, and create outbound data feeds for third-party systems. The position calls for ownership, proactive communication, clear documentation, and collaboration with a US-based enterprise team to deliver reliable solutions aligned with business requirements.

Requirements

At least 5 years of professional experience in data engineering, designing, building, and maintaining production-grade data pipelines.
Strong hands-on experience with Azure Databricks.
Advanced Python skills for data processing, transformation, and pipeline development.
Solid Apache Spark experience, including PySpark and/or Spark SQL, for processing data at scale.
Advanced SQL proficiency, including complex transformations, data validation, and query performance optimization.
Strong knowledge of data warehousing and dimensional modeling, including facts, dimensions, and data marts.
Experience integrating data from multiple sources, including REST APIs, relational databases, cloud storage, and SFTP.
Hands-on experience with Microsoft Azure data services for data ingestion, storage, processing, and integration.
Experience implementing data quality checks and validating consistency between source systems and target data models.
Ability to troubleshoot pipeline failures, investigate data discrepancies, and implement reliable solutions.
Strong problem-solving, analytical, and organizational skills.
Ability to work autonomously, take ownership, and follow established engineering practices and documentation standards.
Strong English communication skills and ability to collaborate effectively with a US-based team.
Strong proficiency in Azure Databricks, Python, and Apache Spark; advanced SQL skills; and solid knowledge of data warehousing and dimensional modeling.
Proven experience developing and maintaining production data pipelines.
Hands-on experience with Microsoft Azure data services and integrating data from multiple enterprise systems.
Strong understanding of data quality, validation, and consistency across source and target systems.
Strong ownership, autonomy, documentation, and communication skills.
Ability to communicate effectively in English and collaborate with a US-based enterprise team.

Nice To Haves

Experience migrating SQL-based data workloads to Azure Databricks.

Responsibilities

Design, build, and maintain production-grade data pipelines using Azure Databricks and Python.
Develop scalable data transformations using PySpark or Spark SQL to integrate data from multiple business systems.
Ingest and process data from APIs, relational databases, cloud storage, and SFTP sources.
Integrate ticketing, CRM, marketing, retail, and other enterprise-platform data into a centralized Data Hub.
Develop and maintain data warehouse structures, dimensional models, and data marts aligned with business requirements.
Write, optimize, and maintain SQL transformations for data accuracy and processing efficiency.
Implement data quality validations and consistency checks between source systems and target models.
Build and maintain outbound data feeds for third-party systems according to defined specifications.
Investigate data discrepancies, troubleshoot pipeline issues, and implement corrective actions.
Document data pipelines, transformation logic, dependencies, and technical processes.
Collaborate with team members and stakeholders to clarify requirements, identify risks early, and deliver reliable data solutions.
Communicate progress, blockers, and potential delays proactively, and propose clear next steps to keep deliverables on track.