Senior Software Engineer - Infrastructure and Tools

Databricks•San Francisco, CA

About The Position

While candidates in the listed locations are encouraged for this role, candidates in other locations will be considered. At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We do this by building and running the world's best data and AI infrastructure platform so our customers can use deep data insights to improve their business. Founded by engineers — and customer obsessed — we leap at every opportunity to solve technical challenges, from designing next-gen UI/UX for interfacing with data to scaling our services and infrastructure across millions of virtual machines. And we're only getting started. As a Senior Software Engineer on the Infrastructure teams, you will build scalable systems to power the Databricks platform, making it the de-facto platform for running Big Data and AI workloads. You will build and extend the Databricks infrastructure platform, which includes multi-cloud systems and services for managing thousands of Kubernetes clusters at scale, storing petabytes of data, highly scalable and distributed API gateways, rate limiting framework, network security and encryption, developer tools and infrastructure (we use Bazel), testing frameworks, and scalable CI/CD framework among many others.

Requirements

BS (or higher) in Computer Science, or a related field
5+ years of experience writing production code in one of: Java, Scala, Go, C++ or Python
Passion for building highly scalable and reliable infrastructure
Experience architecting, developing and deploying large-scale distributed systems at scale
Experience with cloud APIs (e.g., a public cloud such as AWS, Azure, GCP or an advanced private cloud such as Google, Facebook)
Experience with cloud technologies, e.g. AWS, Azure, GCP, Docker, Kubernetes, or Terraform.

Responsibilities

Build and extend components of the core Databricks infrastructure
Architect multi-cloud systems and abstractions to allow the Databricks product to run on top of existing Cloud providers
Improve software development workflows for engineering and operational efficiency.
Use our own data and AI platform (yes!) to analyze build and test logs and metrics to identify areas for improvement.
Develop automated build, test, and release infrastructures.
Set and uphold the standard for engineering processes to support high-quality engineering, including style and code checking, test harnesses, and release packaging.