Databricks Reporting Architect

Movado Group, Inc•Paramus, NJ

9d•$150,000 - $170,000

About The Position

The Databricks Reporting Architect will be responsible for the hands-on design, development, and delivery of enterprise data solutions on the Databricks Lakehouse platform. This role is central to our strategic migration from SAP BW to a modern cloud data architecture, where data flows from SAP systems through middleware into Databricks. The architect will design and implement Medallion Architecture (Bronze/Silver/Gold) pipelines, build high-performance consumption layers, and deliver self-service dashboards and analytics. The role requires strong expertise in Apache Spark, PySpark, SQL, and Python, with a forward-looking approach to leveraging AI-assisted development tools to accelerate delivery. A key focus of this role is balancing platform performance with cost efficiency ensuring compute, storage, and query strategies are optimized for both speed and spend. Collaboration with business stakeholders, SAP functional teams, and data engineering resources is essential to ensure data accuracy and analytical value.

Requirements

Bachelor’s Degree in Computer Science, Engineering, Information Systems, or related field
Minimum 7+ years of experience in data engineering, data warehousing, or analytics architecture
Minimum 3+ years of hands-on experience with Databricks, including Delta Lake, notebooks, and Jobs/Workflows
Strong proficiency in PySpark, spark SQL, and Python for large-scale data transformation
Proven experience designing and implementing Medallion Architecture (Bronze/Silver/Gold) in a production environment
Strong SQL skills for data modeling, query optimization, and reporting layer development
Demonstrated experience building dashboards and reports using Power BI, Tableau, or SAP Analytics Cloud
Experience with AI-assisted development tools (e.g., GitHub Copilot, Databricks AI Assistant) to accelerate data engineering work
Familiarity with SAP data structures and business processes (OTC, P2P), ability to understand data from SAP systems
Experience consuming data from middleware integration layers or data lake ingestion pipelines
Understanding of data governance concepts: Unity Catalog, data lineage, access control
Strong analytical, debugging, and problem-solving skills
Excellent communication and stakeholder management skills; ability to translate business requirements into technical solutions
Ability to work independently in a fast-paced, global environment

Nice To Haves

Master’s Degree or equivalent postgraduate qualification
Experience in offshore/onshore delivery models preferred

Responsibilities

Define how data flows from SAP source systems through middleware into the Databricks Lakehouse, establishing clear guidelines for Medallion Architecture (Bronze/Raw, Silver/Cleansed, Gold/Curated layers)
Build scalable transformation logic using PySpark and Spark SQL, ensuring data quality, performance, and reliability across all layers
Architect optimized storage layers using Delta Lake, ensuring efficient data layout via Z-Ordering, partitioning strategies, and liquid clustering for high-performance queries
Design and manage the Gold/consumption layer to serve reporting and analytics use cases with optimized query performance
Work with Delta Lake for ACID-compliant data storage, time-travel, schema enforcement, and incremental data processing
Translate complex business requirements into scalable physical data models, Dimensional/Star Schema, specifically optimized for Databricks SQL
Serve as the primary architect for data governance, implementing Unity Catalog to enforce fine-grained access control (row and column-level security), data lineage tracking, and centralized auditing
Design automated data quality validation pipelines using Delta Live Tables (DLT) or Great Expectations to ensure data trust across all layers
Implement data quality checks, validation frameworks, and monitoring across all pipeline stages; perform root-cause analysis on data discrepancies
Define and document Databricks architecture patterns, data models, and pipeline designs for reuse, auditability, and compliance
Collaborate with middleware and data engineering teams to consume SAP data (from ECC/S4) landing in Databricks via integration pipelines
Support OData or API-based integrations where data flows between SAP and Databricks; coordinate with upstream teams on data discrepancy resolution
Work with data visualization tools such as SAP Analytics Cloud or Power BI for reporting and analytics
Understand SAP data structures (OTC, P2P domain) to accurately model and transform business data — deep SAP BW development skills not required
Perform root-cause analysis on data discrepancies and resolve data quality issues in collaboration with upstream teams
Build and maintain dashboards and self-service reports for business stakeholders across OTC and P2P processes
Partner with business users to gather requirements, translate them into analytical models, and validate outputs
Actively leverage AI coding tools (e.g. Databricks AI Assistant, Claude) to accelerate development, generate transformation code, and improve code quality
Evaluate and apply emerging AI/ML capabilities available within the Databricks platform (e.g., MLflow, AutoML, Genie) for analytics use cases
Stay current with Databricks platform updates, Apache Spark developments, and modern lakehouse best practices
Support testing, validation, and deployment of solutions with minimal business disruption
Self-manage Business process improvements. Plan milestones and deadlines and provide progress updates.