Databricks Expert

Sprezzatura Management Consulting•Arlington, VA

28d•Remote

About The Position

The Databricks expert builds scalable data pipelines, optimizing workloads, and enabling advanced analytics across cloud platforms, collaborates with data scientists and engineers to transform raw data into actionable insights, ensuring high performance, data integrity, and seamless integration within the Databricks environment. The databricks expert builds robust data lakehouse architectures and enables advanced analytics and machine learning workflows. This role involves collaborating with various stakeholders to understand data requirements and develop solutions.

Requirements

Strong analytical and problem-solving skills.
Proficiency in SQL, Python, R, or other data analysis languages.
Experience with data visualization tools and reporting platforms.
Strong understanding of statistics, data modeling, and analytical techniques.
Similarity with databases, data warehouses, and ETL processes.
Ability to lead and motivate a remote team.
Excellent time management and organizational skills.
Adaptability and openness to new ideas.
Bachelor’s degree in computer science, Information Technology, or a related field. (Master's degree preferred).
5+ years of experience as a data analyst.
Proficiency and experience in SQL, database management systems such as Oracle, SQL Server, MySQL.
Strong knowledge of database structures, theories, principles and practices.
5+ years of experience in big data engineering or analytics.
Experience with cloud platforms, ML, and data cataloguing tools.
Understanding of data privacy and protection regulations.

Responsibilities

Develop and maintain ETL/ELT pipelines using Databricks notebooks and workflows.
Optimize Spark jobs for performance, scalability, and cost-efficiency.
Design and implement data lakehouse architectures.
Integrate Databricks with external data sources, APIs, and on-prem databases.
Collaborate with data scientists, analysts, and engineers to support end-to-end solutions.
Ensure data governance, security, and compliance using tools like Unity Catalog and RBAC.
Mentor junior developer and contribute to engineering best practices.