AI Infrastructure Technical Systems Engineer Hybrid

Cisco•Research Triangle Park, NC

22h•$140,300 - $178,100•Hybrid

About The Position

Cisco IT’s AI Infrastructure team delivers high-performance compute and accelerated AI infrastructure as a service across Cisco’s global enterprise footprint. The team builds and operates enterprise-grade supercomputing platforms supporting foundational models, distributed AI training, inference engines, and machine-learning platforms. This is a hybrid position. The AI Infrastructure Technical Systems Engineer serves as a principal systems architect and technical authority for enterprise AI platforms. The role establishes architectural direction, leads complex infrastructure initiatives, mentors engineering talent, and connects accelerated hardware capabilities with high-performance software execution. The engineer works with a global, multidisciplinary team and regularly interfaces with internal AI researchers, Cisco hardware and network architecture teams, and ecosystem partners including NVIDIA, AMD, and Red Hat.

Requirements

Bachelor’s degree in Computer Science, Engineering, IT, or a related discipline, with 5 or more years of progressive experience in enterprise systems engineering, HPC, AI infrastructure, or SRE.
Proficiency in Python, Go, or Bash, and extensive experience building declarative automation with Ansible, Terraform, CI/CD, and GitOps.
Advanced expertise designing and administering production Kubernetes, Red Hat OpenShift, and HPC workload managers such as Slurm, Kueue, Volcano, and Ray.
Strong hands-on understanding of AI backend network fabrics, RDMA technologies including RoCEv2, and high-speed multi-hundred-gigabit network topologies.
In-depth knowledge of enterprise Linux (RHEL/Ubuntu), kernel module lifecycle, memory subsystems, GPU device drivers, and out-of-band management using Redfish or IPMI.

Nice To Haves

Master’s degree in Computer Science, Computer Engineering, or a related technical discipline, or an additional 2 or more years of equivalent specialized engineering experience.
3 or more years managing enterprise server platforms at scale, such as environments with 500 or more nodes, including automated hardware provisioning and centralized infrastructure management.
2 or more years of hands-on experience deploying, troubleshooting, and optimizing heterogeneous GPU software stacks such as NVIDIA CUDA and AMD ROCm, and running GPU validation suites in production.
3 or more years designing and implementing enterprise secrets management architectures such as HashiCorp Vault and integrating identity federation protocols such as OIDC, SAML, and SSO across multiple production applications.
At least one active, current industry-recognized advanced certification, such as CKA/CKS, RHCA/RHCE, or advanced networking credentials.
Demonstrable active contributions, such as authored commits, merged pull requests, or published technical documentation, to open-source infrastructure projects, Kubernetes SIGs, HPC scheduling communities, or telemetry frameworks.

Responsibilities

Lead architectural design, hardware qualification, capacity planning, and maintenance for large-scale bare-metal NVIDIA and AMD GPU clusters.
Architect resilient, multi-tenant compute and control planes across bare-metal Kubernetes, Red Hat OpenShift, and NVIDIA BCM HPC cluster managers.
Collaborate with network engineering to design, benchmark, and optimize high-throughput, low-latency backend fabrics, including RDMA, RoCEv2, and 400G/800G networks.
Champion Site Reliability Engineering practices to reduce operational toil, enable autonomous self-healing, and uphold Service Level Objectives.
Architect infrastructure-as-code and GitOps deployment pipelines using Ansible, Terraform, Python, Go, and modern CI/CD tooling.
Oversee automation for firmware baselines, BIOS/BMC configuration, kernel driver updates, platform upgrades, and certificate rotations.
Optimize distributed workload scheduling, fair-share queueing, and GPU quota governance across Slurm, Kueue, Volcano, and Ray.
Profile systems performance and diagnose device interconnect bottlenecks, bandwidth utilization, memory hierarchies, and compute efficiency.
Establish automated Day-2 diagnostics and recovery procedures for GPU degradation, driver anomalies, and hardware faults.
Architect end-to-end observability standards using OpenTelemetry, Prometheus, and Splunk, including role-specific dashboards and anomaly detectors.
Act as the senior escalation authority for high-impact incidents, lead root-cause analyses, and align operations with ITIL frameworks.
Author architecture blueprints and standard operating procedures, and mentor and coach mid-level and senior engineers.

Benefits

Medical, dental, and vision insurance
401(k) plan with Cisco matching contribution
Paid parental leave
Short- and long-term disability coverage
Basic life insurance
Cisco restricted stock unit grants may be available, subject to eligibility and vesting conditions.
Paid holidays and floating holiday for eligible non-exempt employees
Birthday day off, year-end holiday shutdown, and personal wellness days
Paid vacation or flexible vacation time off, depending on employee classification
Paid sick time
Additional paid time away may be requested for critical or emergency family issues.
Optional paid volunteer days
Annual bonuses may be available for non-sales roles, subject to Cisco policies.