Sr. Staff Software Engineer, Splunk Security Core AI, Agentic SOC (hybrid)

Cisco•San Jose, CA

1d•Hybrid

About The Position

Build the next generation Agentic SOC for a world where threats are created, adapted, and launched at machine speed. As adversaries increasingly use AI to accelerate attacks, security teams need more than better tools. They need intelligent systems that can reason across complex security data, recognize known and emerging threats, investigate at scale, and work alongside humans to take decisive, governed action. As a Senior Staff Software Engineer on Cisco’s Security Core AI team, you will help turn that vision into reality. You will build the shared platforms, orchestration, and evaluation systems that enable trusted AI agents across Cisco and Splunk security products. Your work will define how agentic security operates across public cloud, on-premises, sovereign cloud, and air-gapped environments, where reliability, control, auditability, and customer trust are non-negotiable. This is an opportunity to help reinvent how security operations work, translating rapid advances in AI into durable capabilities that give defenders the speed and leverage to meet a new generation of threats.

Requirements

6+ years of experience developing, testing, and/or operating production software.
Experience building and/or operating large-scale distributed or AI-enabled production systems, with experience programming with Python or Go.
Bachelors + 10 years of related experience, or Masters + 6 years of related experience, or PhD + 3 years of related experience.

Nice To Haves

Experience with LLM and agentic-system concepts including grounding, planning, tool use, memory, human oversight, evaluation, tracing, and regression analysis.
Demonstrated experience defining AI metrics, building evaluation and experimentation loops, and connecting offline results to production behavior and customer outcomes.
Experience delivering software across public cloud, on-premises, sovereign cloud, or air-gapped environments.
Familiarity with SOC, SIEM, detections, incidents, investigations, response, identity, policy, privacy, and auditability.
Demonstrated success leading cross-team technical initiatives, adopting shared technologies, mentoring senior engineers, and using AI responsibly throughout the software development lifecycle.

Responsibilities

Shape technical direction and lead ambiguous, cross-team Agentic SOC capabilities from concept through measurable production improvement.
Design and build reusable capabilities spanning agent runtime, orchestration, governed tool use, context and memory, evaluation systems, trace analysis, and quality gates.
Translate SOC, SIEM, detection, incident, investigation, and response needs into shared mechanisms in partnership with security-domain experts.
Design agents and supporting systems for grounded reasoning, planning, delegation, human oversight, traceability, auditability, and safe failure handling.
Define measurable success criteria for AI capabilities, including task completion, groundedness, tool-use correctness, trajectory quality, human intervention, safety, latency, cost, and reliability.
Establish closed-loop, metrics-driven development using golden datasets, offline and online evaluations, production traces, SME and customer feedback, experiments, and regression gates.
Connect AI quality metrics to production behavior and customer outcomes, identifying signals that are incomplete, misleading, or vulnerable to optimization without meaningful improvement.
Architect for public cloud, on-premises, sovereign cloud, and air-gapped environments, addressing data residency, restricted connectivity, local model and tool availability, packaging, upgrades, and resource constraints.
Establish patterns for identity propagation, authorization, tenant isolation, policy enforcement, privacy, auditability, and permission-aware actions.
Evaluate, integrate, and drive adoption of technologies across Splunk and Cisco, contributing Security requirements and reusable improvements.
Leverage AI tools throughout requirements, design, coding, review, testing, documentation, debugging, release, and production improvement, with appropriate human validation and security, privacy, and intellectual-property controls.
Diagnose failures spanning models, prompts, tools, data, orchestration, services, infrastructure, and product integrations.
Remain hands-on in implementation and design reviews, raise engineering standards, and mentor and influence senior engineers across teams.

Benefits

medical, dental and vision insurance
a 401(k) plan with a Cisco matching contribution
paid parental leave
short and long-term disability coverage
basic life insurance
10 paid holidays per full calendar year, plus 1 floating holiday for non-exempt employees
1 paid day off for employee’s birthday
paid year-end holiday shutdown
4 paid days off for personal wellness determined by Cisco
16 days of paid vacation time per full calendar year, accrued at rate of 4.92 hours per pay period for full-time employees (non-exempt)
flexible vacation time off program (exempt)
80 hours of sick time off provided on hire date and each January 1st thereafter, and up to 80 hours of unused sick time carried forward from one calendar year to the next
Additional paid time away may be requested to deal with critical or emergency issues for family members
Optional 10 paid days per full calendar year to volunteer
annual bonuses (non-sales roles)
performance-based incentive pay (sales roles)
grants of Cisco restricted stock units