Software Engineer

Cisco•Research Triangle Park, NC

3d•$128,600 - $184,900•Remote

About The Position

As a Software Engineer on the Foundational Model Service (FMS) team, you will be responsible for maintaining and improving the platform that serves foundational models (small, large language, and embedding models) to engineering teams across Cisco. This involves serving models on Kubernetes clusters using runtimes like Nim and Vllm, as well as benchmarking, evaluating, and monitoring models for releases. The platform operates under a 99.9% uptime SLA, requiring a focus on reliability and operational excellence. Your role will include supporting model onboarding and releases, participating in an on-call rotation, automating validation processes, and enhancing monitoring capabilities. Additionally, you will apply Agentic solutions to proactively identify and resolve operational issues. You will develop software aligned with Cisco Design Thinking Principles, emphasizing simplification and user experience, while adhering to secure coding practices, protecting user privacy, and following software development best practices. Collaboration with design, product management, and other engineering teams will be key to building effective customer solutions. You will be responsible for creating technical design documentation, contributing to end-user documentation, and debugging platform issues in both development and production environments. This role is ideal for a systems or infrastructure engineer with experience in LLMs, understanding of inference serving, and a proactive approach to learning and problem-solving. You will work alongside experienced engineers and gain deep insights into AI infrastructure development, model runtimes, and large-scale product delivery. On-call duties are shared among the team.

Requirements

Bachelor’s degree in Computer Science, Information Systems, or a related field with 3 years of related experience; master’s degree with 1 year of related experience; PhD; or equivalent practical work experience.
Experience developing and operating production services on Kubernetes and Linux, including exposure to GPU based AI infrastructure.
Hands-on experience serving models with vLLM, NVIDIA NIM, Triton, or a comparable inference runtime, including building supporting services.
Experience leading technical work, mentoring engineers, documenting systems, and supporting critical services through an on-call rotation.

Nice To Haves

Experience evaluating and benchmarking models to support production release decisions.
Experience operating GPU workloads with NVIDIA CUDA or AMD ROCm.
Familiarity with distributed inference, disaggregated serving, KV cache aware routing, capacity planning, or cost optimization.
Experience fine tuning transformer models or evaluating RAG and agent systems using relevant frameworks.

Responsibilities

Keep the Foundational Model Service platform running and improve its ease of operation.
Support model onboarding and releases.
Take turns on call for the platform.
Help automate the validation processes for deployments.
Contribute to the monitoring systems that track service behavior.
Apply Agentic solutions to operations for early problem detection and automatic remediation.
Develop software consistent with Cisco Design Thinking Principles, focusing on simplification and user experience.
Utilize secure coding practices and protect user privacy.
Follow software development best practices.
Partner with design, product management, and other engineering teams to build solutions.
Create technical design documentation.
Contribute to end-user documentation.
Debug platform issues in development and production.
Evaluate and benchmark models to support production release decisions.
Operate GPU workloads with NVIDIA CUDA or AMD ROCm.
Work with distributed inference, disaggregated serving, KV cache aware routing, capacity planning, or cost optimization.
Fine tune transformer models or evaluate RAG and agent systems using relevant frameworks.

Benefits

medical, dental and vision insurance
a 401(k) plan with a Cisco matching contribution
paid parental leave
short and long-term disability coverage
basic life insurance
10 paid holidays per full calendar year
1 floating holiday for non-exempt employees
1 paid day off for employee’s birthday
paid year-end holiday shutdown
4 paid days off for personal wellness
16 days of paid vacation time per full calendar year (non-exempt employees)
flexible vacation time off program (exempt employees)
80 hours of sick time off provided on hire date and each January 1st thereafter
up to 80 hours of unused sick time carried forward
Additional paid time away may be requested to deal with critical or emergency issues for family members
Optional 10 paid days per full calendar year to volunteer
annual bonuses (for non-sales roles)
performance-based incentive pay (for sales roles)