Databricks Administrator/Architect

$145,600 - $158,080/Yr

Corelleaf Design Studio - Raleigh, NC

Full-time - Mid Level

Raleigh, NC

About the position

The Databricks Administrator/Architect role involves the design, implementation, and maintenance of the Databricks environment for the client. This position requires collaboration with internal staff to ensure the platform meets business needs, while also providing mentorship and support to team members. The role emphasizes scalability, performance, and security of the Databricks clusters, along with the development of tailored data solutions.

Responsibilities

• Provide mentorship, guidance, overall knowledge share, and support to team members, promoting continuous learning and development.
• Oversee the design, implementation, and maintenance of Databricks clusters.
• Ensure the platform's scalability, performance, and security.
• Provide escalated support and troubleshooting to users.
• Oversee maintenance of role-based access to data and features in the Databricks Platform using Unity Catalog.
• Review clusters health check and best practices implementation.
• Review and maintain documentation for users and administrators.
• Design and implement tailored data solutions to meet customer needs and use cases, spanning from ingesting data from APIs, building data pipelines, analytics, and beyond within a dynamically evolving technical stack.
• Work on projects involving on-prem data ingestion into Azure using ADF.
• Build data pipelines based on the medallion architecture that clean, transform, and aggregate data from disparate sources.

Requirements

• Extensive hands-on experience implementing Lakehouse architecture using Databricks Data Engineering platform, SQL Analytics, Delta Lake, Unity Catalog - 5 Years
• Strong understanding of Relational & Dimensional modeling - 5 Years
• Demonstrate proficiency in coding skills - Python, SQL, and PySpark to efficiently prioritize performance, security, scalability, robust data integrations - 6 Years
• Experience implementing serverless real-time/near real-time architecture using Cloud (i.e., Azure, AWS, or GCP Tech Stack), and Spark technology (Streaming & ML) - 2 Years
• Experience with Azure Infrastructure configuration (Networking, architect and build large data ingestion pipelines and conducting data migrations using ADF or similar technology) - 4 Years
• Experience working with SQL Server features such as SSIS and CDC - 7 Years
• Experience with Databricks platform, security features, Unity Catalog, and data access control mechanisms - 2 Years
• Experience with GIT code versioning software - 4 Years

Nice-to-haves

• Databricks Certifications

Benefits

• Competitive hourly pay of $70 - $76
• Opportunity for mentorship and professional development
• Hybrid work environment with in-person requirements