Senior QA Engineer
About The Position
The Senior QA Engineer will help develop quality engineering capabilities for Splunk and Cisco’s Enterprise Security Apps organization, supporting an AI-agentic Security Operations Center (SOC). The role focuses on making complex systems reliable, performant, and secure through benchmarking, evaluation strategies, and scalable testing infrastructure. The engineer will contribute to technical performance goals and the reliability, security, and integrity of Cisco’s unified security ecosystems. The role includes developing evaluation frameworks for AI agents, workflows, and system processes; building AI-based solutions or agents to mitigate risk; and implementing proof-of-concepts according to experimentation and validation standards. The engineer will also use advanced validation techniques, including LLM-as-judge methodologies, tenant isolation probes, and behavioral regression suites, while participating in code and design reviews and technical collaboration.
Requirements
- Bachelor’s degree.
- 8+ years of experience in software development and test automation, including building large-scale testing ecosystems.
- Programming experience with Python and Groovy, including designing scalable, maintainable, and high-performance testing infrastructure.
- Experience working with security compliance frameworks.
- Experience implementing modern performance testing methodologies.
- Experience using AI to build automation, including writing AI skills to build test automation.
Nice To Haves
- Experience translating quality challenges into structured and actionable technical implementations.
- Experience applying modern tooling within the AI/ML landscape to drive detection efficacy and system reliability.
- Experience with test automation tools and frameworks, including contract testing frameworks and Playwright.
- Experience with security and compliance tools such as OCSF validation libraries, vacuum linter, Caldera, tenant isolation probes, and audit log validation.
- Experience in AI/ML testing using LLM APIs such as Claude and Codex, prompt engineering, agentic workflows, LangChain, and LangSmith.
Responsibilities
- Develop foundations and evaluation frameworks for assessing AI agents, workflows, and system processes.
- Develop AI-based solutions or agents to mitigate risk and enhance system integrity across testing methodologies.
- Collaborate with team members to implement benchmarking protocols and design scalable testing infrastructure.
- Support technical performance goals and improve engineering efficiency through evaluation strategies and implementation.
- Participate in code and design reviews, technical collaboration, and validation of technical hypotheses.
- Implement proof-of-concepts while following standards for experimentation and validation of testing paradigms.
- Execute advanced validation techniques, including LLM-as-judge methodologies, tenant isolation probes, and behavioral regression suites.
Benefits
- Medical insurance
- Dental insurance
- Vision insurance
- 401(k) plan with Cisco matching contribution
- Paid parental leave
- Short- and long-term disability coverage
- Basic life insurance
- Paid holidays
- Floating holiday for non-exempt employees
- Paid birthday day off
- Paid year-end holiday shutdown
- Personal wellness days
- Paid vacation or flexible vacation time off, depending on employee classification
- Sick time off
- Additional paid time away may be requested for critical or emergency family issues
- Optional paid volunteer days
- Annual bonuses for eligible non-sales roles
- Potential Cisco restricted stock unit grants
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
