Staff Test & Release Engineer, Inference Cloud

d-Matrix • Santa Clara, CA

1d • $155,000 - $235,000

About The Position

The Inference Cloud Engineering team builds deployment and orchestration technology for d-Matrix’s inference services, including an internal cloud for testing and tuning models and a customer-facing cloud with self-service inference endpoints and compute. The accelerators run in on-premises clusters, while parts of the internal cloud also run in Azure.

The Staff Test & Release Engineer owns end-to-end testing and software releases for this platform. The role ensures that automated tests verify features against user stories on the hardware where they will run, and that releases can be delivered predictably and rolled back safely. Testing must account for workloads that span d-Matrix Corsair accelerators and GPUs, which are scarce and shared. The engineer will build on existing test suites, CI pipelines, and benchmarks, collaborating with QE, product, platform, and infrastructure teams to create a scalable test and release system without slowing development. This is not a manual testing, gatekeeping, company-wide CI/CD, or performance engineering role.

Requirements

• At least 6 years of experience in test infrastructure, software engineering in test, release engineering, or a closely related role, including ownership of end-to-end testing or releases for a distributed system.
• A track record of building test or release automation that measurably improved release reliability or speed.
• Experience operating test and release infrastructure across on-premises clusters and a public cloud; Azure experience is preferred, including AKS, Azure DevOps or GitHub Actions on Azure, and Azure networking and identity.
• Strong Kubernetes experience, including deploying and debugging multi-service workloads, Helm, operators and custom resources, and automated test-cluster setup and teardown.
• Experience designing CI/CD pipelines such as GitLab CI, GitHub Actions, Buildkite, or Argo, including artifact versioning, environment promotion, and rollback.
• Proficiency in Python and pytest or a comparable framework, plus fluency in a second language sufficient to read and test code being qualified; much of the orchestration layer is Rust.
• Experience testing on shared, limited hardware such as GPU or accelerator fleets, accounting for scheduling, contention, and hardware faults.
• Sound judgment about what to test end to end versus lower in the stack, and ability to explain tradeoffs to code owners.
• A record of building consensus across teams and turning input into a strategy that teams follow.
• Fluency with AI and agentic tools for writing tests, triaging failures, and searching logs and code, with the discipline to verify their output.

Nice To Haves

• Experience testing or releasing LLM serving systems such as vLLM, SGLang, or TensorRT-LLM, including accuracy evaluations such as GSM8K, HumanEval, or MMLU and performance benchmarks as release gates.
• Familiarity with speculative decoding, disaggregated serving, or heterogeneous inference.
• Hardware-in-the-loop testing experience, including qualifying software across driver, firmware, and hardware versions.
• Experience with operator and controller testing tools such as envtest, kuttl, or chainsaw, and with upgrade and CRD version-migration testing.
• Experience shipping software into customer-managed, on-premises Kubernetes environments, including compatibility matrices and customer installation packaging.
• Experience with canary or progressive rollouts for customer-facing services.
• Rust experience.
• Startup experience or comfort in environments where priorities shift quickly.

Responsibilities

• Design and build end-to-end test infrastructure that provisions homogeneous and heterogeneous inference deployments on demand, tests them, and tears them down.
• Translate user stories and acceptance criteria into automated end-to-end tests, including failure cases such as invalid API keys, unavailable endpoints, and failed draft nodes.
• Integrate GPU and Corsair hardware into CI, schedule fairly across a limited shared pool, maintain test-signal quality, address flaky tests, and route failures to the appropriate owners.
• Own release strategy for the orchestration platform, Kubernetes tooling, and benchmarking tools, including versioning, cadence, branching, promotion, rollback, release notes, and go/no-go criteria.
• Lead the transition from platform-wide releases to independently released components, defining component boundaries, cadence, versioning, and compatibility and upgrade testing with platform architects.
• Define release gates for functional tests, upgrade and compatibility tests, model accuracy evaluations, and performance baselines such as time to first token, time per output token, and throughput.
• Partner with QE, product, platform, and infrastructure teams on a shared test strategy and advocate for testable acceptance criteria.
• Enable feature engineers to write and own tests for their work.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed