Sr Databricks Engineer

Hitachi • Dallas, TX

17h

About The Position

We are looking for a highly skilled Databricks Solution Architect to lead the design and implementation of scalable, enterprise-grade data platforms using Databricks. The ideal candidate will combine strong technical expertise in data engineering and cloud platforms (AWS/Azure/GCP) with architectural leadership, solution design capability, and strong stakeholder engagement skills.

Requirements

• Strong hands-on experience with Databricks (clusters, notebooks, Delta Lake, MLflow, Unity Catalog).
• Experience with at least one cloud provider (AWS, Azure, GCP).
• Strong proficiency in Spark, Python, SQL, and distributed data processing.
• Experience designing large-scale data solutions including ingestion, transformation, storage, and analytics.
• Experience with streaming technologies (Structured Streaming, Kafka, Kinesis, EventHub).
• CI/CD practices for data pipelines (Azure DevOps, GitHub Actions, Jenkins, etc.).
• Strong communication skills with the ability to engage both technical and business teams.
• Experience working in Agile environments.
• Ability to simplify complex technical concepts for non-technical audiences.
• Strong analytical, problem-solving, and decision-making abilities.

Nice To Haves

• Databricks Certified Data Engineer Professional / Architect certification.
• AWS/Azure/GCP cloud architect certifications.
• Experience with BI tools (Tableau, Power BI, Looker).
• Experience in machine learning workflows and ML operations.
• Background in large-scale data modernization or cloud migration projects.

Responsibilities

• Design end-to-end data architectures using Databricks Lakehouse Platform.
• Architect scalable ETL/ELT pipelines, real-time streaming solutions, and advanced analytics platforms.
• Define data models, storage strategies, and integration patterns aligned with business and enterprise architecture standards.
• Provide guidance on cluster configuration, performance optimization, cost management, and workspace governance.
• Lead technical discussions and design workshops with engineering teams and business stakeholders.
• Provide best practices, frameworks, and reusable component designs for consistent delivery.
• Perform code reviews and provide technical mentoring to data engineers and developers.
• Collaborate with product owners, business leaders, and analytics teams to translate business requirements into scalable technical solutions.
• Create and present solution proposals, architectural diagrams, and implementation strategies.
• Support pre-sales or discovery phases with technical input when needed.
• Define and implement governance standards across Databricks workspaces (data lineage, cataloging, access control, etc.).
• Ensure compliance with regulatory and organizational security frameworks.
• Implement best practices for monitoring, auditing, and data quality management.
• Stay updated on Databricks features, roadmap, and industry trends.
• Recommend improvements, optimizations, and modernization opportunities across the data ecosystem.
• Evaluate integration of complementary technologies (Delta Live Tables, MLflow, Unity Catalog, streaming frameworks, etc.).

Benefits

• Industry-leading benefits, support, and services that look after your holistic health and wellbeing.
• Flexible arrangements that work for you (role and location dependent).

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed