QA Software Engineer NC Hybrid
About The Position
Cisco is hiring an Automation QA Engineer for its AI and Automation portfolio, supporting the Circuit platform, a central AI engine used to power productivity and process automation across Cisco. The team works at the intersection of applied research and production, with a focus on performance, reliability, and functional quality for more than 80,000 monthly users. This is a hybrid position. The role owns and advances Circuit’s automated testing ecosystem, replacing manual regression processes with AI-driven, fully automated test suites for new and legacy features. The engineer designs and maintains end-to-end, API, and performance automation; integrates tests into CI/CD pipelines; and uses AI-assisted coding tools to generate and maintain tests while validating their stability, correctness, and security compliance. The role also develops approaches for testing AI and LLM features and agent workflows, reports quality metrics and risks, and leads automation standards and collaboration across engineering, product, and UX teams.
Requirements
- Bachelor’s degree plus 5 or more years, master’s degree plus 3 or more years, or PhD plus 1 or more year of related experience in Computer Science, AI/ML, or a related technical field.
- Experience in test automation and regression framework development, with a strong understanding of functional quality assurance.
- Experience with Playwright (TypeScript/JavaScript) for end-to-end/UI automation and/or Python with Pytest or Robot Framework; familiarity with Selenium.
- Experience with REST API testing; familiarity with API contract testing.
- Solid practical and conceptual knowledge of software development lifecycles, CI/CD pipelines such as GitHub Actions or Jenkins, and automated testing best practices.
- Experience with AI-assisted coding tools such as GitHub Copilot, Cursor, or Claude Code, plus critical thinking and Agile/Git workflows to ensure high-quality, scalable test automation.
Nice To Haves
- Experience testing AI/ML-powered web applications, including LLM-based features, chatbots, or AI agent workflows.
- Hands-on performance/load testing experience with k6 or JMeter and familiarity with AI/LLM performance metrics such as TTFT, inter-token latency, and token throughput.
- Experience in AI Ops/MLOps or building or fine-tuning custom LLMs.
- Experience with observability and reporting tools such as Splunk, Grafana, or Prometheus.
- Familiarity with Docker and Kubernetes for test environment management and with Google Cloud Platform (GCP).
- Track record of leading technical projects and mentoring junior engineers.
- Strong communication and collaboration skills, including ability to influence cross-functional stakeholders.
- Experience building and productizing agentic applications with a focus on evaluation and business acceptance.
- GraphQL testing experience.
Responsibilities
- Architect, build, and maintain automated regression and end-to-end test suites for the Circuit platform using Playwright, Python/Pytest, and Selenium.
- Design and automate REST and GraphQL API tests covering functional correctness, contract validation, error handling, and performance, including AI/model-serving endpoints under concurrent load.
- Plan and execute load, stress, and soak tests using k6 or JMeter; track response-time percentiles, throughput, error rates, and AI-specific performance metrics.
- Use AI agents and AI-assisted coding tools to generate and maintain regression suites for new and legacy features; assess generated code for stability and security compliance before merging.
- Develop testing approaches for AI/LLM features, including model response quality, behavior consistency, and end-to-end AI agent workflows across reasoning and action layers.
- Integrate automated tests into CI/CD pipelines and build dashboards and reports; communicate quality metrics, risk assessments, and defect trends to engineering leadership.
- Lead automation initiatives by defining standards, mentoring peers, and collaborating with development, product, and UX teams to embed automation-first testing in the development lifecycle.
Benefits
- Medical insurance
- Dental insurance
- Vision insurance
- 401(k) plan with Cisco matching contribution
- Paid parental leave
- Short- and long-term disability coverage
- Basic life insurance
- Paid holidays
- Floating holiday for non-exempt employees
- Paid birthday day off
- Paid year-end holiday shutdown
- Personal wellness days
- Paid vacation or flexible vacation time off, depending on employee classification
- Sick time off
- Additional paid time away may be requested for critical or emergency family issues
- Optional paid volunteer days
- Annual bonuses for eligible non-sales employees
- Potential Cisco restricted stock unit grants
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
