About The Position

We are seeking a highly experienced Systems Analyst / Databricks Administrator to support enterprise data and analytics initiatives. This role involves administering and optimizing the Databricks platform to enable scalable data engineering, analytics, and AI/ML workloads. The ideal candidate will bring strong technical expertise, problem-solving skills, and the ability to work independently in a fast-paced environment.

Requirements

  • 8+ years of experience in systems analysis or platform administration
  • Strong hands-on experience with Databricks (AWS environment)
  • Expertise in cluster configuration, job orchestration, and workspace management
  • Proficiency in Apache Spark (performance tuning & troubleshooting)
  • Experience with IAM, SCIM, RBAC for access control
  • Knowledge of data security, governance, and compliance frameworks
  • Experience integrating with cloud storage solutions (S3 or similar)
  • Familiarity with Databricks SQL, notebooks, and workflows
  • Experience with monitoring tools and performance optimization
  • Exposure to DevOps tools (Terraform, CI/CD pipelines, scripting)

Nice To Haves

  • Experience in enterprise or government environments
  • Knowledge of Unity Catalog for governance and access control
  • Understanding of cost optimization strategies for Databricks
  • Experience supporting AI/ML workloads (MLflow, Databricks ML)
  • Familiarity with data lake / lakehouse architectures
  • Programming knowledge in Python, SQL, or Scala

Responsibilities

  • Administer and manage Databricks workspaces in a cloud environment (AWS)
  • Configure and maintain clusters, job scheduling, and workspace settings
  • Manage user access, roles, and permissions using IAM, SCIM, and RBAC
  • Implement and enforce cluster policies and governance standards
  • Monitor platform performance, availability, and health
  • Integrate Databricks with cloud storage services (e.g., S3)
  • Support data engineers, analysts, and data scientists
  • Perform performance tuning and troubleshooting using Apache Spark
  • Ensure data security, compliance, and encryption standards
  • Automate processes using DevOps tools (Terraform, CI/CD, scripting)
  • Maintain documentation and provide technical guidance to teams
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service