Software Engineer, Compute Performance

Cisco • Milpitas, CA

1d • $155,000 - $222,600

About The Position

The Cisco Validated Infrastructure Services (CVIS) team establishes and proves the performance of large AI clusters before they are handed over to customers. This involves deploying each cluster, validating it against performance baselines, and delivering evidence of its readiness for service. The platform supporting this work is full stack, integrating workflow and API services, network and infrastructure automation, and a UI for running validations across labs and customer environments. As a Software Engineer on this team, you will contribute to analyzing and improving the performance of AI compute systems, encompassing hardware, system software, and application workloads.

Requirements

• Bachelor’s degree with + 5 years of related experience, or Master’s degree with + 3 years of related experience, or PhD with relevant research experience, or equivalent related work experience.
• Experience debugging, testing, or tuning Linux-based systems.
• Experience running or evaluating AI, machine learning, or HPC workloads.
• 3 + years of experience with a programming or scripting language such as Python, C, C++, Go, or Bash.

Nice To Haves

• Familiarity with GPU computing technologies such as CUDA, ROCm, NCCL, or RCCL.
• Familiarity with performance tools such as Nsight, DCGM, rocprofiler, or similar tools.
• Understanding of GPU system components such as memory, NUMA, PCIe, and scale-up interconnects.
• Exposure to NVIDIA NVLink and NVSwitch or AMD Infinity Fabric/xGMI.
• Experience with Kubernetes, Slurm, or other distributed workload environments.
• Strong ownership, documentation, and cross-team collaboration skills.

Responsibilities

• Run and analyze AI training, inference, and HPC workloads on GPU-accelerated systems.
• Investigate compute, memory, PCIe, storage, and workload scaling issues.
• Use telemetry, logs, traces, and profiling tools to identify performance bottlenecks.
• Develop automation for performance testing, regression detection, and failure triage.
• Work with hardware and software teams to resolve system and workload performance issues.
• Document findings, test procedures, and performance results.

Benefits

• medical insurance
• dental insurance
• vision insurance
• 401(k) plan with a Cisco matching contribution
• paid parental leave
• short-term disability coverage
• long-term disability coverage
• basic life insurance
• paid holidays
• floating holiday
• paid day off for employee’s birthday
• paid year-end holiday shutdown
• paid days off for personal wellness
• paid vacation time
• flexible vacation time off program
• sick time off
• paid time away to volunteer
• annual bonuses
• performance-based incentive pay

What This Job Offers

• Job Type: Full-time
• Career Level: Senior