ML Engineer - Infrastructure

Flexion Robotics•San Francisco, CA

2d•Onsite

About The Position

At Flexion, we are building the autonomy stack for humanoid robots. Our mission is to drive the transition from fragile prototypes to real-world deployments of humanoids. We were founded by leading scientists in robot reinforcement learning (ex-Nvidia, ex-ETH Zürich) and backed by leading international VC firms. In just months, we went from our first line of code to deploying real humanoid capabilities with our customers, leveraging simulation and reinforcement learning. Today, we are rapidly expanding the capabilities of our autonomy stack, our customer base, and our team. We are looking for an experienced ML engineer to join Flexion’s experienced infrastructure team and take ownership of Flexion’s GPU compute platforms. This is a senior, on-site role with significant scope. At Flexion, we are building the brain for humanoid robots, which involves training foundation models with vast amounts of data on large GPU clusters. You will own the design, bring-up, operation and optimization of performant clusters. You will work with AI engineers to help them optimize their training speed and hardware utilization. You will also influence strategic compute planning and contribute to new tools and platforms for iterating on our AI models efficiently. This will put you at the heart of Flexion’s AI development and allow you to directly impact the execution of our ambitious roadmap. You will closely collaborate with the company’s leadership, engineers of the infrastructure team and AI engineers across the company.

Requirements

Degree in Computer Science, Electrical Engineering or Software Engineering (or equivalent practical experience) plus significant industry experience.
Hands-on experience with the training or inference of large models (billions of parameters) on distributed multi-node GPU hardware. This can include bringing up and running the cluster, writing and optimizing training/inference code, building ML pipelines, etc.
Proficiency in Python and working knowledge of PyTorch.
Deep understanding of distributed training concepts (DDP, FSDP, NCCL).
Experience with at least one cloud platform (AWS, GCP, Azure or neoclouds) or large-scale on-premises GPU infrastructure.
Experience with job scheduling and orchestration tools: Slurm and/or Kubernetes/KubeRay.

Nice To Haves

Familiarity with profilers (e.g., PyTorch Profiler, Dynolog, HTA, Nsight).
Experience with high-performance or parallel file systems (e.g., Lustre).
Experience provisioning compute on multiple cloud providers.
Experience with infrastructure-as-code and configuration management (Terraform, Ansible).

Responsibilities

Architect, run and continuously improve existing and future cloud-based GPU clusters.
Select the best frameworks and tooling to run our clusters efficiently.
Work on cluster provisioning, job schedulers and monitoring systems.
Help AI engineers optimize their training workloads and maximize hardware utilization using profilers, contributing to our core ML libraries.
Contribute to short- and long-term GPU compute strategies in collaboration with our AI engineering teams and help execute on them.
Optimize capacity and cost by exploring multi-cloud strategies and evaluating trade-offs.
Raise the bar on engineering practices, including testing, code quality, documentation, and system reliability.

Benefits

Competitive Compensation
Joining a leading robotics team & exposure to never-done-before research
Energetic, collaborative culture with a bias for action and regular community events
Enhanced pension plan
Relocation & permit sponsorship
Enhanced holiday & paid leave perks
Central Zürich office with top-tier robotics testing facilities and infrastructure
401(k) with company contributions
Health, dental & vision coverage with the flexibility to choose your own plan
Open PTO policy & paid company holidays