Databricks Architect - Tarrytown, NY [Hybrid]

OMG Technology • Village of Tarrytown, NY

8d • Hybrid

About The Position

We are looking for a Databricks Architect to lead enterprise-scale Lakehouse architecture and implementation on AWS. The role involves designing scalable data solutions, modernizing existing data platforms, establishing governance standards, and leading architecture discussions with customers. The position is hybrid in Tarrytown, NY, with three days onsite per week, for a one-year contract.

Requirements

• 15+ years of overall Data & Analytics experience.
• 4+ years of hands-on Databricks experience.
• Strong experience with Databricks, Delta Lake, Unity Catalog, DLT, and Auto Loader.
• Proficiency in PySpark, Spark SQL, Python, and Structured Streaming.
• AWS experience with S3, IAM, Glue, Redshift, EMR, and Lambda.
• Experience with Airflow, CDC, and enterprise data migration.
• Knowledge of data governance, security, and observability.
• Strong customer-facing, solution architecture, and technical leadership skills.

Nice To Haves

• Healthcare or pharmaceutical domain experience is preferred but not mandatory.
• Experience with enterprise-scale data platform modernization and migration is beneficial.

Responsibilities

• Lead enterprise-scale Databricks Lakehouse architecture and implementation on AWS.
• Drive migration of EMR, NiFi, Redshift, and legacy ETL workloads to Databricks.
• Evaluate Databricks capabilities, including Serverless, Lakehouse Federation, Genie, Databricks Apps, Lakebase, and AI/ML.
• Architect scalable batch, streaming, CDC, and real-time data solutions.
• Design reusable frameworks using Delta Lake, PySpark, Auto Loader, DLT, and Databricks Workflows.
• Define governance, security, lineage, and data-quality standards using Unity Catalog.
• Establish best practices for performance, scalability, and cost optimization.
• Lead customer workshops, architecture reviews, and solution assessments.
• Define target-state architecture, migration roadmaps, and reusable patterns.

What This Job Offers

• Career Level: Senior
• Education Level: No Education Listed