QA Software Engineer NC Hybrid

Cisco•Research Triangle Park, NC

4d•$128,600 - $184,900•Hybrid

About The Position

The Automation QA Engineer will own and advance the automated testing ecosystem for Cisco’s Circuit platform, an AI engine used for productivity and process automation. The role focuses on keeping the platform reliable, scalable, production-ready, and functionally sound for its users. It will transform regression testing from manual processes to AI-driven automation for new and legacy features, while reducing technical debt and supporting releases. The engineer will design and maintain end-to-end, API, and performance test automation; build scalable test frameworks; and integrate tests into CI/CD pipelines to provide continuous quality feedback. The role also uses AI-assisted coding tools to generate and maintain tests, while reviewing generated code for stability, correctness, and compliance with security standards. It includes validation of AI/LLM features and agent workflows, reporting quality metrics and risks, and leading automation practices in collaboration with engineering, product, and UX teams. This is a hybrid position.

Requirements

Bachelor’s degree and 4+ years, Master’s degree and 3+ years, or PhD and 1+ year of related experience in Computer Science, AI/ML, or a related technical field.
Experience in test automation and regression framework development, with a deep understanding of functional quality assurance.
Experience with Playwright (TypeScript/JavaScript) for E2E/UI automation and/or Python with Pytest or Robot Framework; familiarity with Selenium.
Experience with REST API testing; familiarity with API contract testing.
Solid conceptual and practical knowledge of software development lifecycles, CI/CD pipelines such as GitHub Actions or Jenkins, and automated testing best practices.
Experience with AI-assisted coding tools such as GitHub Copilot, Cursor, or Claude Code, along with critical thinking and Agile/Git workflows to ensure high-quality, scalable test automation.

Nice To Haves

Experience testing AI/ML-powered web applications, including LLM-based features, chatbots, or AI agent workflows.
Hands-on experience with performance/load testing tools such as k6 or JMeter and AI/LLM metrics such as TTFT, inter-token latency, and token throughput.
Experience in AI Ops/MLOps or building or fine-tuning custom LLMs.
Experience with observability and reporting tools such as Splunk, Grafana, or Prometheus.
Familiarity with Docker and Kubernetes for test environment management and with Google Cloud Platform (GCP).
Proven track record of leading technical projects and mentoring junior engineers.
Strong communication and collaboration skills, with the ability to influence cross-functional stakeholders.
Experience building and productizing agentic applications with a focus on evaluation and business acceptance.
GraphQL testing experience.

Responsibilities

Architect, build, and maintain automated regression and end-to-end test suites for the Circuit platform using Playwright, Python/Pytest, and Selenium.
Design and automate REST and GraphQL API tests for functional correctness, contract validation, error handling, and performance, including AI/model-serving endpoints under concurrent load.
Plan and execute load, stress, and soak tests using k6 or JMeter; track response-time percentiles, throughput, error rates, and AI-specific performance metrics.
Use AI agents and AI-assisted coding tools to generate and maintain regression tests for new and legacy features; assess generated code for stability and security compliance before merge.
Develop testing approaches for AI/LLM features, including model response quality, behavioral consistency, and end-to-end agent workflows across reasoning and action layers.
Integrate automated tests into CI/CD pipelines such as GitHub Actions or Jenkins for continuous validation.
Build quality dashboards and reports using tools such as Grafana and Allure, and communicate quality metrics, risk assessments, and defect trends to engineering leadership.
Lead automation initiatives by defining standards, mentoring peers, and collaborating with development, product, and UX teams to embed automation-first testing into the development lifecycle.

Benefits

Medical insurance
Dental insurance
Vision insurance
401(k) plan with Cisco matching contribution
Paid parental leave
Short- and long-term disability coverage
Basic life insurance
Cisco restricted stock unit grants may be available, subject to eligibility and continued employment
10 paid holidays per full calendar year
One floating holiday for non-exempt employees
Paid day off for employee’s birthday
Paid year-end holiday shutdown
Four paid personal wellness days
Paid vacation or flexible vacation time off, depending on employee classification
80 hours of sick time off provided on hire date and each January 1st, with up to 80 unused hours carried forward
Additional paid time away may be requested for critical or emergency family issues
Optional 10 paid volunteer days per calendar year
Annual bonuses may be available for non-sales roles, subject to Cisco policies