Senior Staff Software Engineer, AI Runtime

Databricks • San Francisco, CA

19h

About The Position

Databricks is hiring a Senior Staff Software Engineer to work on the systems behind AI Runtime, its serverless platform for GPU training on enterprise data. The role will address complex technical challenges and help shape the product’s future, including its architecture, reliability, developer experience, and support for evolving GPUs and workloads.

The engineer will lead technical direction for AI training infrastructure, solve difficult multi-node training problems, and improve training speed and cost across model architectures and GPU accelerators. They will collaborate with research, product, and customers to set priorities, align teams on technical decisions, champion engineering excellence, and help hire and develop engineers.

Requirements

• 15+ years of experience in large-scale distributed systems, with significant depth in GPU training, high-performance computing, or ML systems.
• Hands-on experience training large models with FSDP, DeepSpeed, or Megatron, and fluency in parallelism strategies.
• Expertise in GPU performance, including accelerator architecture, high-speed interconnects, and kernels.
• Strong understanding of training resilience patterns, including checkpointing, failure detection, and automatic recovery.
• Background building and operating multi-tenant cloud platforms as products.
• Track record of setting technical direction across teams and delivering business impact.
• Ability to bring teams to agreement on difficult trade-offs and a reputation as a mentor sought out by senior engineers.
• BS in Computer Science or a related field.

Nice To Haves

• MS or PhD.

Responsibilities

• Drive AI Runtime’s architecture and lead Databricks’ long-term technical direction in AI training infrastructure.
• Define the reliability and observability strategy for a growing and increasingly complex fleet.
• Shape the developer experience for people and agents so large training jobs are simple to launch and manage.
• Anticipate industry trends and select technical investments that prepare AI Runtime for new GPU generations and workload types.
• Solve difficult multi-node training problems involving orchestration, GPU scheduling, data loading, and checkpointing.
• Improve training speed and cost across model architectures and GPU accelerators.
• Work with research, product, and customers to determine what to build and align teams on technical decisions.
• Champion engineering excellence, raise the engineering bar, and help hire and grow engineers.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior