Sr. Databricks Lead/Architect
About The Position
Job Title : Sr. Databricks Lead/Architect Location: Boston, MA (5 days onsite) Duration: 12 Months Note: Need some local to Boston area. Key Responsibilities Customer & Stakeholder Engagement Act as the primary technical point of contact for customer stakeholders (Business, Product Owners, Architects, and IT Leadership). Lead requirements discovery workshops, solution discussions, and architecture reviews. Translate business requirements into scalable Azure Databricks based data architectures. Present architecture options, trade offs, and roadmaps to senior stakeholders and decision makers. Provide technical leadership during program inception, design sign off, and critical milestones. Solution & Data Architecture Design end to end data platforms using Azure Databricks, Azure Data Lake (ADLS), Delta Lake, and surrounding Azure services. Define lake house architectures supporting batch, streaming, and near real time use cases. Establish data modeling standards (bronze/silver/gold layers, dimensional models, curated data marts). Ensure solutions align with security, governance, performance, and cost optimization best practices. Define integration patterns with upstream and downstream systems (APIs, Kafka/Event Hub, Power BI, Azure Synapse, ML workloads). Support cloud migration, modernization, and legacy platform replacement initiatives. Review and validate offshore team deliverables for architecture adherence, quality, and performance. Resolve technical blockers and guide te ams on complex implementations. Enforce coding standards, CI/CD practices, and reusable frameworks. Ensure ontime, high quality delivery aligned with customer expectations. Drive DevOps enablement (Azure DevOps, Git, CI/CD pipelines). Establish monitoring, logging, and cost optimization strategies. Required Skills & Experience Technical Skills Strong hands on expertise with Azure Databricks (Spark, Delta Lake, Unity Catalog). Deep experience with: Azure Data Lake Storage (ADLS Gen2) Azure Data Factory / Synapse Pipelines Streaming (Kafka / Event Hub / Structured Streaming) Python, SQL, Spark optimization Solid understanding of data warehousing, lakehouse, and analytics architectures. Experience with Power BI or other BI/analytics tools. Familiarity with ML/AI integration with Databricks is a plus. Leadership & Soft Skills Proven experience working directly with customers in an onshore role. Strong communication, presentation, and stakeholder management skills. Experience managing distributed / offshore delivery teams. Ability to balance hands on architecture work with leadership responsibilities. Strong ownership mindset and decision making ability. Experience & Qualifications 10+ years in data engineering / data architecture roles. 5+ years designing solutions on Azure data platforms. 3+ years in a technical lead or solution architect role. Azure or Databricks certifications preferred.
Requirements
- Strong hands on expertise with Azure Databricks (Spark, Delta Lake, Unity Catalog).
- Deep experience with: Azure Data Lake Storage (ADLS Gen2) Azure Data Factory / Synapse Pipelines Streaming (Kafka / Event Hub / Structured Streaming) Python, SQL, Spark optimization
- Solid understanding of data warehousing, lakehouse, and analytics architectures.
- Experience with Power BI or other BI/analytics tools.
- Proven experience working directly with customers in an onshore role.
- Strong communication, presentation, and stakeholder management skills.
- Experience managing distributed / offshore delivery teams.
- Ability to balance hands on architecture work with leadership responsibilities.
- Strong ownership mindset and decision making ability.
- 10+ years in data engineering / data architecture roles.
- 5+ years designing solutions on Azure data platforms.
- 3+ years in a technical lead or solution architect role.
Nice To Haves
- Familiarity with ML/AI integration with Databricks is a plus.
- Azure or Databricks certifications preferred.
Responsibilities
- Act as the primary technical point of contact for customer stakeholders (Business, Product Owners, Architects, and IT Leadership).
- Lead requirements discovery workshops, solution discussions, and architecture reviews.
- Translate business requirements into scalable Azure Databricks based data architectures.
- Present architecture options, trade offs, and roadmaps to senior stakeholders and decision makers.
- Provide technical leadership during program inception, design sign off, and critical milestones.
- Design end to end data platforms using Azure Databricks, Azure Data Lake (ADLS), Delta Lake, and surrounding Azure services.
- Define lake house architectures supporting batch, streaming, and near real time use cases.
- Establish data modeling standards (bronze/silver/gold layers, dimensional models, curated data marts).
- Ensure solutions align with security, governance, performance, and cost optimization best practices.
- Define integration patterns with upstream and downstream systems (APIs, Kafka/Event Hub, Power BI, Azure Synapse, ML workloads).
- Support cloud migration, modernization, and legacy platform replacement initiatives.
- Review and validate offshore team deliverables for architecture adherence, quality, and performance.
- Resolve technical blockers and guide te ams on complex implementations.
- Enforce coding standards, CI/CD practices, and reusable frameworks.
- Ensure ontime, high quality delivery aligned with customer expectations.
- Drive DevOps enablement (Azure DevOps, Git, CI/CD pipelines).
- Establish monitoring, logging, and cost optimization strategies.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Education Level
No Education Listed
