Databricks Platform Architect

Guidehouse•Arlington, VA

About The Position

Architect and manage Databricks clusters, workspaces, and integrations with cloud services (Azure/AWS/GCP). Implement CI/CD pipelines for Databricks notebooks and jobs. Optimize cluster configurations for cost and performance. Ensure compliance with security and governance standards. Collaborate with data engineers and scientists to support data pipelines and ML workflows. Conduct ETL and data quality analysis using various technologies (i.e., Python, Databricks). Maintain good working relationships with accounts/clients to enhance customer satisfaction Ensure data governance and quality assurance standards are met. Organize and lead meetings, including scheduling meetings; drafting and delivering agendas and meeting minutes; providing and archiving required documentation; and documenting, tracking, and following up on action items. Summarize and present information and reports to the team and make recommendations (both oral and written).

Requirements

Bachelor's degree is required
Minimum SEVEN (7) years of total experience in cloud-based data platforms
Minimum FIVE (5) years experience with Databricks.
Strong knowledge of Spark architecture and distributed computing.
Hands-on experience with Terraform or other IaC tools.
Experience with Unity Catalog and Delta Lake.
Familiarity with Kubernetes and container orchestration.
Strong scripting skills (Python, Bash).
Excellent problem-solving skills and attention to detail.
Strong communication and collaboration skills, with the ability to work effectively in a team environment.

Nice To Haves

Master’s degree
Databricks Certified Data Engineer Professional
Azure/AWS/GCP Cloud Architect or Administrator.
Terraform Associate
Experience working with system and application logs.
Experience with the design and build of ETL programs, interfaces, and data reconciliation processes, preferably in Databricks.
An ability to obtain a federal security clearance

Responsibilities

Architect and manage Databricks clusters, workspaces, and integrations with cloud services (Azure/AWS/GCP).
Implement CI/CD pipelines for Databricks notebooks and jobs.
Optimize cluster configurations for cost and performance.
Ensure compliance with security and governance standards.
Collaborate with data engineers and scientists to support data pipelines and ML workflows.
Conduct ETL and data quality analysis using various technologies (i.e., Python, Databricks).
Maintain good working relationships with accounts/clients to enhance customer satisfaction
Ensure data governance and quality assurance standards are met.
Organize and lead meetings, including scheduling meetings; drafting and delivering agendas and meeting minutes; providing and archiving required documentation; and documenting, tracking, and following up on action items.
Summarize and present information and reports to the team and make recommendations (both oral and written).

Benefits

Medical, Rx, Dental & Vision Insurance
Personal and Family Sick Time & Company Paid Holidays
Parental Leave
401(k) Retirement Plan
Group Term Life and Travel Assistance
Voluntary Life and AD&D Insurance
Health Savings Account, Health Care & Dependent Care Flexible Spending Accounts
Transit and Parking Commuter Benefits
Short-Term & Long-Term Disability
Tuition Reimbursement, Personal Development, Certifications & Learning Opportunities
Employee Referral Program
Corporate Sponsored Events & Community Outreach
Care.com annual membership
Employee Assistance Program
Supplemental Benefits via Corestream (Critical Care, Hospital Indemnity, Accident Insurance, Legal Assistance and ID theft protection, etc.)
Position may be eligible for a discretionary variable incentive bonus