Senior Platform Software Engineer – AI/ML & GPU Infrastructure

Cisco•San Jose, CA

19h

About The Position

The Senior Platform Software Engineer will join Cisco’s Platform Engineering team to build and operate the AI Compute Platform, the GPU infrastructure supporting model training, inference, and LLM workloads at scale. The team builds and runs a unified control plane and Kubernetes platform, manages GPU capacity across shared and dedicated pools, and partners with infrastructure, hardware, and engineering teams to provide a reliable self-service platform. In this foundational role, the engineer will own the architecture, technical direction, and operation of control-plane and scheduling systems for GPU infrastructure. The work spans distributed control planes, Kubernetes, GPU acceleration, and workload orchestration. The engineer will develop distributed systems, services, and APIs for multi-cluster environments, tenant GPU quotas, and complex training and inference workloads, while providing technical leadership, mentoring, and operational direction to improve reliability and throughput across the AI compute fleet.

Requirements

Bachelor’s degree in Computer Science, Computer Engineering, or equivalent practical experience.
8+ years of software engineering experience building, architecting, or extending Kubernetes-based platforms or large-scale distributed systems, including technical leadership experience.
Strong software development proficiency in Go or Python.
Demonstrated experience architecting platform control-plane services, custom APIs, Kubernetes Controllers, Operators, or CRDs.
Hands-on experience building, architecting, or operating batch and accelerator schedulers for GPU or AI/ML workloads, such as Kueue, Volcano, Slurm, or custom scheduler plugins.
Deep experience with GPU scheduling, multi-tenant GPU infrastructure, or workloads on accelerated compute platforms.
Track record designing for reliability and operational excellence using Infrastructure as Code such as Terraform, GitOps, CI/CD, and cloud-native observability tools such as Splunk, Prometheus, or Grafana.

Nice To Haves

Experience scaling Kubernetes control-plane internals, such as API server, etcd, controller-manager, or scheduler framework, or multi-cluster federation at large node and object counts.
Familiarity with modern data-center GPU architectures such as NVIDIA H100, H200, B200/B300, or AMD Instinct accelerators, including scheduling, placement, and interconnect considerations such as NVLink and InfiniBand.
Familiarity with topology-aware placement, collective-communication libraries such as NCCL, and gang scheduling for large-scale multi-node training clusters.
Experience supporting AI/ML infrastructure, including model-serving pipelines, distributed training, workload orchestration, or MLOps platforms.
Low-level systems engineering experience, such as Linux kernel tuning, cgroups, or eBPF.
Experience leading ambiguous platform initiatives end-to-end and representing platform architecture in cross-team design review forums.
Experience participating in production infrastructure on-call rotations, incident response, and postmortem or runbook processes.

Responsibilities

Architect and develop platform control-plane services, APIs, and controllers for multi-cluster lifecycle management, tenant project onboarding, and GPU quota enforcement across shared and dedicated pools.
Own the technical architecture for GPU capacity scheduling and allocation, including fractional GPU allocation, multi-tenant isolation, and workload-aware scheduling for training and inference.
Design and develop automated capacity-intake, demand-classification, and request-routing tools to direct workloads to appropriate clusters and compute pools.
Architect automated remediation tools to detect node and workload failures, trigger remediation, and optimize system-wide performance.
Design, build, and scale reliable Kubernetes-based platforms using OpenShift or upstream Kubernetes.
Lead design reviews and set technical standards for platform services, Kubernetes Operators, Controllers, CRDs, and automation built with Go and Python.
Mentor engineers and raise standards for cloud-native software engineering and platform reliability.
Drive Infrastructure as Code, GitOps, and CI/CD automation for the platform.
Participate in on-call rotation, incident response, and postmortem reviews.

Benefits

Medical insurance
Dental insurance
Vision insurance
401(k) plan with Cisco matching contribution
Paid parental leave
Short- and long-term disability coverage
Basic life insurance
Paid holidays
Floating holiday for non-exempt employees
Paid day off for employee’s birthday
Paid year-end holiday shutdown
Paid personal wellness days
Flexible vacation time off for exempt employees
Paid vacation time for non-exempt employees
Paid sick time
Additional paid time away may be requested for critical or emergency family issues
Optional paid volunteer days
Annual bonuses for eligible non-sales employees
Potential Cisco restricted stock unit grants