Software Development Engineer — GPU Fleet Management & AI Infrastructure

Advanced Micro Devices, Inc•San Jose, CA

16h•Hybrid

About The Position

AMD is looking for an experienced software engineer to help build Fleet Manager, a secure control plane for operating large-scale AMD GPU infrastructure. Fleet Manager provides scheduling, workload orchestration, hardware health management, interactive development environments, and model-serving capabilities across GPU clusters. You will design and build production systems spanning distributed control planes, Kubernetes, GPU scheduling, inference infrastructure, and developer-facing APIs and tools. You will join a team working at the intersection of systems software, cloud infrastructure, and accelerated computing. Your work will directly influence how engineers and customers train, serve, debug, and operate workloads on current and future AMD GPU platforms.

Requirements

Strong systems-software development experience in Rust, C++, Go, or a comparable language. Production Rust experience is highly desirable.
Experience designing and operating distributed systems, control planes, schedulers, or cloud infrastructure.
Strong understanding of concurrency, asynchronous programming, state machines, and failure recovery.
Experience building reliable services using REST, streaming, WebSocket, or gRPC APIs.
Experience with Kubernetes internals, controllers, operators, scheduling, resource management, or custom resources.
Familiarity with workload scheduling technologies such as Kueue, JobSet, Slurm, or other batch and cluster schedulers.
Experience with PostgreSQL-backed services, schema evolution, transactions, leader election, and optimistic concurrency.
Experience developing command-line tools and stable, user-focused APIs.
Understanding of container security, multi-tenant isolation, authentication, authorization, and secrets management.
Experience with production observability, including metrics, structured logging, tracing, alerting, and incident diagnosis.
Ability to write high-quality, maintainable code with careful attention to correctness, testing, and operational behavior.
Experience with source control, continuous integration, automated testing, profiling, and debugging tools.
Demonstrated ability to lead technically challenging projects and collaborate across organizational boundaries.
Effective written and verbal communication skills.
Bachelor’s or Master’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related field, or equivalent practical experience.

Nice To Haves

Experience in one or more of the following areas would be beneficial but is not required: AMD GPU architecture, ROCm, HIP, amd-smi, RCCL, or GPU device plugins
Distributed AI training and multi-node collective communication
Large-model inference using platforms such as vLLM, SGLang, PyTorch, or similar runtimes
GPU topology, capacity management, performance analysis, and hardware diagnostics
Kubernetes networking, storage, admission control, and workload isolation
OpenAI-compatible inference APIs, request routing, streaming, and rate limiting
High-performance shared storage and large model or checkpoint management
Bare-metal, virtualized, and cloud GPU infrastructure
Production security and threat modeling for multi-tenant compute platforms

Responsibilities

Design and develop Fleet Manager’s distributed control-plane services, APIs, schedulers, inference gateway, and command-line tools.
Build reliable orchestration for GPU training, inference, custom jobs, and interactive development workloads.
Develop scalable scheduling and admission-control capabilities, including priority, fairness, topology-aware placement, quotas, backfilling, and multi-node workload coordination.
Implement durable reconciliation, lifecycle management, retries, idempotency, and recovery across PostgreSQL and external execution systems.
Integrate Fleet Manager with Kubernetes and technologies such as Kueue, JobSet, container runtimes, storage systems, and observability platforms.
Help evolve Fleet Manager into a portable orchestration layer capable of supporting Kubernetes, Slurm, Spur, and future execution environments.
Develop GPU health, diagnostics, quarantine, and controlled-remediation capabilities using ROCm and AMD hardware telemetry.
Build secure multi-tenant infrastructure with strong authentication, authorization, workload isolation, auditing, rate limiting, and least-privilege defaults.
Improve the reliability and performance of AI inference services, including routing, streaming, load shedding, health detection, and usage metering.
Define and maintain stable APIs, data models, compatibility contracts, and operational procedures.
Diagnose complex failures across distributed services, Kubernetes, networking, storage, GPU runtimes, drivers, and hardware.
Develop automated unit, integration, failure-injection, and production-readiness tests.
Work with AMD architecture, driver, platform, security, and machine-learning software teams to enable current and future GPU products.
Participate in new GPU, system, cluster, and software-stack bring-up.
Provide technical leadership through design reviews, code reviews, mentoring, and cross-functional problem solving.