Databricks Administrator

Cayuse Holdings•Cedar Park, TX

7d•Hybrid

About The Position

The Databricks Administrator is responsible for the design, implementation, configuration, administration, and ongoing support of Databricks environments, with a focus on optimizing performance, security, reliability, and cost effectiveness. The administrator will work closely with data engineers, data scientists, cybersecurity personnel, and other IT stakeholders to ensure the integrity, availability, and confidentiality of enterprise data and analytics workloads. This position aligns with Cayuse’s core values of Innovation, Excellence, Collaboration, Adaptability, and Integrity by fostering technical solutions that meet customer needs, promoting teamwork, and prioritizing quality in deliverables.

Requirements

8 years of experience administering Databricks workspaces in a cloud environment, specifically AWS.
8 years of strong understanding of Databricks cluster configuration, job scheduling, and workspace management.
8 years of experience managing user access, roles, and permissions using IAM, SCIM, and role-based access control (RBAC).
8 years of proficiency with Apache Spark concepts, including performance tuning and troubleshooting.
8 years of experience integrating Databricks with cloud storage services (e.g., S3).
8 years of experience implementing and enforcing cluster policies and workspace governance standards.
8 years of familiarity with Databricks SQL, notebooks, and job orchestration.
8 years of experience monitoring platform health, performance, and availability.
8 years of understanding of data security, encryption, and compliance requirements.
8 years of experience with DevOps or automation tools such as Terraform, CI/CD pipelines, and scripting.
Must be able to pass a background check. May require additional background checks as required by projects and/or clients at any time during employment.
Exceptional interpersonal skills with the ability to communicate in a clear, professional, and articulate manner.
Exceptional verbal and written communication skills.
Excellent organizational, analytical, and problem-solving skills with high-level attention to detail.
Ability to analyze systems and procedures
Strong multitasking skills with the ability to manage multiple design streams across concurrent work effort.
Must be self-motivated and able to work well independently as well as on a multi-functional team.
Ability to handle sensitive and confidential information appropriately.

Nice To Haves

4 years of experience administering Databricks in an enterprise or government environment.
4 years of experience with Databricks Unity Catalog for data governance and access control.
4 years of knowledge of cost management and optimization for Databricks workloads.
4 years of experience supporting AI/ML workloads using Databricks ML and MLflow.
4 years of familiarity with data lake and lakehouse architecture.
4 years of knowledge of Python, SQL, or Scala for administration and troubleshooting.

Responsibilities

Administer, configure, and support the Databricks platform to enable scalable data engineering, analytics, and AI/ML workloads for the enterprise.
Ensure platform reliability, security, cost optimization, and compliance with relevant standards and organizational policies.
Provide technical support, guidance, and troubleshooting assistance to data engineers, analysts, and data scientists across various business units.
Manage workspace configuration, cluster policies, and user access controls to maintain secure and efficient operation of the Databricks environment.
Integrate Databricks with cloud services and related enterprise platforms to enable seamless data and analytic workflows.
Proactively monitor platform performance, usage, and system health, implementing corrective actions and improvements as necessary.
Collaborate with stakeholders to understand business objectives, operational challenges, and user requirements related to data analytics and platform utilization.
Identify and evaluate alternative solutions, conducting studies and cost/benefit analyses to determine the best approach for organizational needs.
Analyze user requirements, procedures, and problems to automate processing or improve existing computer systems.
Engage with personnel from multiple organizational units to assess current operational procedures, identify system deficiencies, and gather input/output requirements such as data input forms and reporting formats.
Document detailed user requirements, program functions, and necessary steps for the development or modification of computer programs supporting the Databricks platform.
Review computer system capabilities, specifications, and scheduling limitations to assess the feasibility of requested program changes within existing system constraints.
Other duties as assigned.