Principal Machine Learning Engineer
About The Position
Cisco’s CX AI Incubation and Agents team develops the foundational AI platform used by application and agent teams across Cisco to create intelligent customer experiences. The platform supports cloud, on-premises, and fully air-gapped customer environments and includes model and inference layers, evaluation systems, agentic and orchestration infrastructure, foundational AI services and APIs, and packaging and upgrade capabilities. The team works across classical machine learning, domain-specific language models, and Agentic AI, and develops solutions in areas such as delivery intelligence, network automation, and edge computing. As a Principal Machine Learning Engineer, you will lead ambiguous, high-impact cross-functional work from architecture and design through production and customer deployment. You will serve as the architectural and technical authority for designing, building, and scaling the AI foundation behind Cisco IQ and evolving customer needs. This is a hands-on technical leadership role, setting standards for system design, platform governance, and engineering excellence across the CX engineering ecosystem.
Requirements
- Bachelor’s degree plus 14 years of related experience, or master’s degree plus 11 years of related experience, or Ph.D. plus 7 years of related experience.
- Experience developing, fine-tuning, and optimizing models in Python using PyTorch or JAX, with techniques such as PEFT or LoRA, quantization, and distillation.
- Experience architecting and deploying production LLM and/or SLM inference runtimes optimized with continuous batching, KV-cache management, or speculative decoding.
- Experience architecting and deploying production RAG systems and agentic workflows using vector databases.
- Experience designing and implementing automated AI evaluation frameworks, including golden test datasets, LLM-as-a-judge methodologies, drift detection, and/or CI/CD regression gates.
- Experience architecting production AI platform APIs with robust quota management and versioning, and deploying across hybrid cloud and air-gapped environments.
Nice To Haves
- Recognized industry contributions through publications, patents, conference presentations, or core contributions to open-source AI/ML projects.
- Hands-on experience optimizing inference workloads for resource-constrained compute environments, including CPU, NPU, small GPU footprints, and on-premises hardware appliances.
Responsibilities
- Design and build robust, multi-tenant AI services with scalable APIs, versioning, backward compatibility, and quota management across cloud, on-premises, and air-gapped environments.
- Select, evaluate, and implement models, frameworks, inference runtimes, and orchestration technologies.
- Translate emerging AI capabilities into production architectures optimized for scalability, latency, memory, and compute across GPU, CPU, and edge environments.
- Design and implement AI workload cost optimizations, including improvements to token efficiency, model selection, inference performance, and compute utilization.
- Establish measurable engineering standards for cost efficiency.
- Lead architecture and design reviews and resolve complex technical trade-offs.
- Mentor senior engineers and influence platform direction through hands-on technical leadership and execution.
- Lead cross-functional engagements from architecture and design through production and customer deployment.
Benefits
- Medical, dental, and vision insurance, subject to plan eligibility rules.
- 401(k) plan with a Cisco matching contribution.
- Paid parental leave.
- Short- and long-term disability coverage.
- Basic life insurance.
- Potential eligibility for Cisco restricted stock unit grants, which vest following continued employment for defined periods.
- Paid holidays and a floating holiday for non-exempt employees.
- Paid birthday leave, a year-end holiday shutdown, and paid personal wellness days.
- Vacation time for non-exempt employees; exempt employees participate in a flexible vacation time-off program with no defined limit, subject to availability and business limitations.
- Sick time, with unused sick time carryover subject to the stated limit.
- Additional paid time away may be requested for critical or emergency family issues.
- Optional paid volunteer days.
- Annual bonuses for eligible non-sales employees, subject to Cisco policies.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Principal
