QA Software Engineer NC Hybrid

Cisco•Rtp, NC

3d•$128,600 - $184,900•Hybrid

About The Position

Cisco’s AI and Automation portfolio team builds Circuit, a central AI engine that powers productivity and process automation across Cisco. The team works at the intersection of applied research and production, with a focus on performance, reliability, and functional quality for more than 80,000 monthly users. This hybrid role owns and advances the automated testing ecosystem that keeps Circuit reliable, scalable, and production-ready across releases. The Automation QA Engineer will transform regression testing beyond manual processes by building AI-driven automated test suites for new and legacy features. The role designs and owns end-to-end, API, and performance test automation; builds scalable frameworks; integrates testing into CI/CD pipelines; and uses AI-assisted coding tools to generate and maintain tests while validating their stability, correctness, and security compliance. The position also validates AI/LLM features and agent workflows, tracks quality and performance metrics, and leads automation standards in collaboration with engineering, product, and UX teams.

Requirements

Bachelor’s degree with at least 4 years, master’s degree with at least 3 years, or PhD with at least 1 year of related experience in Computer Science, AI/ML, or a related technical field.
Experience in test automation and regression framework development, with a strong understanding of functional quality assurance.
Experience with Playwright (TypeScript/JavaScript) for E2E/UI automation and/or Python with Pytest or Robot Framework; familiarity with Selenium.
Experience with REST API testing; familiarity with API contract testing. GraphQL testing is a plus.
Practical and conceptual knowledge of software development lifecycles, CI/CD pipelines such as GitHub Actions or Jenkins, and automated testing best practices.
Experience with AI-assisted coding tools such as GitHub Copilot, Cursor, or Claude Code, plus critical thinking and Agile/Git workflows to deliver high-quality, scalable test automation.

Nice To Haves

Experience testing AI/ML-powered web applications, including LLM-based features, chatbots, or AI agent workflows.
Hands-on experience with performance/load testing using k6 or JMeter and AI/LLM performance metrics such as TTFT, inter-token latency, and token throughput.
Experience in AI Ops/MLOps or building and fine-tuning custom LLMs.
Experience with observability and reporting tools such as Splunk, Grafana, or Prometheus.
Familiarity with Docker and Kubernetes for test environment management and with Google Cloud Platform (GCP).
Track record of leading technical projects and mentoring junior engineers.
Strong communication and collaboration skills, including the ability to influence cross-functional stakeholders.
Experience building and productizing agentic applications with a focus on evaluation and business acceptance.

Responsibilities

Architect, build, and maintain automated regression and end-to-end test suites for the Circuit platform using Playwright, TypeScript/JavaScript, Python/Pytest, and Selenium.
Design and automate REST and GraphQL API tests for functional correctness, contract validation, error handling, and performance, including AI/model-serving endpoints under concurrent load.
Plan and run load, stress, and soak tests using k6 or JMeter; track response-time percentiles, throughput, error rates, Time to First Token, and inter-token latency.
Use AI agents and AI-assisted coding tools to generate and maintain tests for new and legacy features; assess generated code for stability and ensure it meets security standards before merging.
Develop testing approaches for AI/LLM features, including model response quality, behavior consistency, and end-to-end AI agent workflows across reasoning and action layers.
Integrate automated tests into CI/CD pipelines such as GitHub Actions and Jenkins; build dashboards and reports using tools such as Grafana and Allure.
Communicate quality metrics, risk assessments, and defect trends to engineering leadership.
Lead automation initiatives as a team captain by defining standards, mentoring peers, and collaborating with development, product, and UX teams to embed automation-first testing in the development lifecycle.

Benefits

Medical insurance
Dental insurance
Vision insurance
401(k) plan with Cisco matching contribution
Paid parental leave
Short- and long-term disability coverage
Basic life insurance
Restricted stock unit grants may be available, subject to eligibility and continued employment
Paid holidays
Floating holiday for non-exempt employees
Paid day off for employee’s birthday
Paid year-end holiday shutdown
Paid personal wellness days
Paid vacation or flexible vacation time off, depending on employee classification
Sick time off
Additional paid time away may be requested for critical or emergency family issues
Optional paid volunteer days
Annual bonuses for non-sales roles, subject to Cisco policies
Performance-based incentive pay for employees on sales plans, subject to the applicable Cisco plan