Data Engineer – Databricks

BuzzClan LLC•Beaverton, OR

2d•Hybrid

About The Position

We are seeking an experienced Data Engineer with strong Databricks expertise to design, develop, and maintain scalable enterprise data pipelines and data engineering solutions. The ideal candidate will have hands-on experience with Databricks, PySpark, Spark SQL, Delta Lake, Python, and SQL, along with experience in cloud platforms and modern data engineering practices. Candidates must be local to or able to work onsite in Beaverton, Oregon 3 days per week.

Requirements

6+ years of overall IT experience, including 5+ years of Data Engineering experience.
Strong hands-on experience with Databricks.
Strong expertise in PySpark, Spark SQL, and Delta Lake.
Strong programming skills in Python and SQL.
Proven experience building scalable ETL/ELT pipelines.
Experience with Azure, AWS, or GCP, preferably Azure.
Experience with Azure Data Factory (ADF) and/or Airflow.
Strong understanding of data modeling and data warehousing.
Experience with data pipeline and query performance optimization.
Hands-on experience with Git, CI/CD, and DevOps practices.
Strong analytical, problem-solving, collaboration, and communication skills.

Nice To Haves

Delta Live Tables (DLT)
Unity Catalog
Structured Streaming
Azure cloud data services
Experience working with large-scale enterprise data platforms.

Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines and enterprise data solutions.
Develop and optimize data processing solutions using Databricks, PySpark, Spark SQL, and Delta Lake.
Build reliable and high-performance data pipelines using Python and SQL.
Implement data integration and transformation processes across various enterprise data sources.
Work with cloud platforms such as Azure, AWS, or GCP, with preference for Azure.
Develop and manage data workflows using orchestration tools such as Azure Data Factory (ADF) and/or Apache Airflow.
Apply data modeling, data warehousing, and performance optimization best practices.
Implement data quality, scalability, reliability, and performance improvements across data platforms.
Use Git, CI/CD, and DevOps practices to support automated and reliable data deployments.
Collaborate with business stakeholders, data architects, developers, and other cross-functional teams to deliver enterprise data solutions.
Troubleshoot complex data pipeline and processing issues and provide effective technical solutions.
Contribute to modern Databricks capabilities such as Delta Live Tables (DLT), Unity Catalog, and Structured Streaming where applicable.