Senior Systems Engineer, Test Frameworks & Validation Platform

CoreWeave•Sunnyvale, NY

12d•Onsite

About The Position

The Systems Engineering team owns the host software stack that turns a freshly provisioned bare-metal machine into a healthy Kubernetes worker — the OS image, kernel and drivers, and the hardware and firmware beneath them. We build and operate the test framework that qualifies every server image on real physical hardware before it reaches one of the largest GPU fleets in the world, and we're extending that framework into HPC verification, Slurm-on-Kubernetes, and further down the stack. As a Senior Software Engineer on the Systems Engineering team, you will own and evolve our test framework — in-house, Kubernetes-native system that qualifies host images and infrastructure on real hardware. You'll harden the framework's core, broaden what it can validate, and keep CI fast and trustworthy at fleet scale, where a 0.1% failure rate is thousands of GPUs. Day-to-day, you'll build test harnesses and CI infrastructure, extend coverage into HPC and Slurm-on-Kubernetes (SUNK), and shape the abstractions other engineers use to add their own tests without reinventing the framework underneath them.

Requirements

3+ years of experience building test infrastructure, systems software, or platform tooling at scale, with real ownership of frameworks or automation for low-level software.
Fluent in Python, with either proven Rust experience or a strong systems background (Go, C, C++) and a genuine appetite to build in Rust — the framework's core is Rust, and you'll live in it.
Comfortable operating in a Kubernetes environment and reasoning about how software is built, containerized, deployed, and tested.
Solid Linux systems background — the boot chain, kernel and drivers, low-level debugging — and comfort working close to the hardware.
Real testing discipline: you've built automation that proves systems work, and you have strong opinions about flakiness, hermeticity, and signal.
Clear communicator who treats the test framework as a product other engineers want to use, not a chore they route around.

Nice To Haves

Rust and Kubernetes-native workflow orchestration (e.g., Argo Workflows) experience.
HPC or large-cluster experience — InfiniBand/RoCE, GPU/accelerator validation, or performance-regression frameworks.
Slurm or Slurm-on-Kubernetes (SUNK) experience.
Firmware or lower-level hardware validation experience.
Applying LLMs to test workflows — triage, failure classification, flaky-test detection.
Contributions to open-source test frameworks, Rust crates, or systems projects.

Responsibilities

Own and extend the test framework — that pins to a real host, boots the configuration under test, runs a containerized check, validates it, and records a structured result.
Broaden what the framework qualifies: HPC and fabric verification, Slurm-on-Kubernetes (SUNK), and further down the stack into firmware and hardware, building the abstractions that make new coverage easy to add.
Keep the pipeline fast, hermetic, and trustworthy — boot consistency, per-host resource locking, and ruthless flaky-test elimination — so engineers trust the signal.
Own the results and reporting path end to end: structured test results, storage, dashboards, and the triage surfaces engineers use to turn a red run into a root cause quickly.
Explore AI-native testing — LLM-driven log triage, failure classification, and regression detection — as the fleet and test surface grow.
Collaborate with firmware, kernel, imaging, and HPC teams to embed testing into their release process.