Member of Technical Staff — Reliability-CI Infrastructure

RadixArk•Palo Alto, CA

20h

About The Position

RadixArk is hiring a Member of Technical Staff to own and architect the CI and infrastructure that supports SGLang, an open-source LLM inference engine. The CI system runs more than 300 GPU tests across NVIDIA, AMD, Intel, and Ascend hardware pools and gates commits for a project with more than 100 contributors. The role is responsible for keeping CI trustworthy, fast, reliable, and secure, working directly with core maintainers, hardware partners, and the open-source community. The engineer will improve CI reliability and speed, develop regression-based detection in place of static thresholds, harden runners against supply-chain risks, and improve contributor feedback. The position calls for treating CI infrastructure as a system to be deliberately designed, rather than simply maintaining pipeline configuration. The expected annual salary range is $200,000–$400,000 USD, plus equity, depending on background, skills, and experience.

Requirements

At least 3 years of experience operating CI/CD at scale using GitHub Actions, Buildkite, Jenkins, GitLab CI, or similar.
Deep knowledge of Linux, Docker, and GPU computing.
Experience managing self-hosted runners.
Strong Bash and Python skills.
A security mindset, including knowledge of CI supply-chain risks, fork pull request attack vectors, and runner hardening.
Experience with NVIDIA GPU drivers, CUDA, NCCL, and InfiniBand/RDMA in CI contexts.
Familiarity with ML inference workloads, including model loading, KV cache, and quantization.

Nice To Haves

Experience with CI for a large open-source project with 100 or more contributors.
Experience with AMD ROCm or Intel XPU CI pipelines.

Responsibilities

Own CI reliability end-to-end: triage failures, distinguish real regressions from flaky tests and infrastructure issues, and keep the main branch green.
Build regression-based CI by replacing hardcoded static thresholds with automated baseline comparison, including a metrics pipeline, durable storage, and detection logic.
Harden runner infrastructure with ephemeral runners, container isolation, and security protections for fork pull request execution.
Reduce CI time by right-sizing evaluation suites, deduplicating server startups, and separating pull request smoke tests from nightly full runs.
Improve developer experience through faster feedback, clearer failure messages, and workflow orchestration.
Work directly with core maintainers, hardware partners, and the open-source community to maintain a trustworthy, fast, and secure CI system.
Take over daily CI triage, fix recurring flaky tests, and reduce pull request CI time.
Deploy regression-based checks on nightly CI and prototype ephemeral runners with runner isolation.
Aim for zero flaky tests and fully green main CI when there is no real regression.