Staff Site Reliability Engineer (SRE) (Hybrid)
About The Position
The Splunk Agent Resilience team is defining the future of AI resilience. Together our team provides scalable, cost-effective evaluation and guardrails that ensure AI agents behave as intended, improving reliability and reducing risks. This unified approach empowers our customers to expertly deploy and manage AI-powered applications with enhanced observability and control. As a Staff Site Reliability Engineer (SRE), you will provide technical leadership for the reliability, scalability, and operational architecture of Splunk Agent Observability's platform. You will define the long-term reliability strategy, lead major infrastructure initiatives, and drive engineering excellence across deployment automation, production operations, and platform resiliency. In addition to owning complex production systems, you will influence engineering direction across teams and establish best practices for operating large-scale cloud and on-prem deployments.
Requirements
- 8+ years’ experience with a Bachelor's degree or 6+ yrs with Masters or 3+ years with a PhD, or equivalent related experience; Experience to include at least 6 years in Site Reliability Engineering, Platform / Cloud / Infrastructure Engineering, or related fields.
- 5+ years’ operating large-scale Kubernetes platforms in production.
- Experience designing highly available, scalable, and resilient distributed systems.
- Experience with AWS, GCP, or other public cloud platforms.
- Strong experience designing CI/CD platforms and deployment automation at scale.
Nice To Haves
- Expertise in observability, monitoring, alerting, capacity planning, and performance engineering.
- Strong programming skills in Python and/or Go
- Deep experience with Infrastructure as Code (Terraform or similar)
- Experience with MLOps preferred
- Strong understanding of networking, distributed systems, storage, databases, and cloud architecture
- Experience operating both SaaS and enterprise/on-prem deployments
- Demonstrated technical leadership across multiple engineering teams
- Experience leading incident management, postmortems, and long-term reliability initiatives
Responsibilities
- Define and drive the technical roadmap for platform reliability, scalability, and operational excellence.
- Lead the architecture and evolution of deployment platforms supporting cloud and air-gapped customer environments.
- Establish reliability engineering standards, including service level objectives (SLOs), operational readiness, capacity planning, and resiliency reviews.
- Lead major reliability and scalability initiatives across Kubernetes, deployment infrastructure, databases, and networking.
- Drive automation to eliminate operational toil and improve engineering productivity.
- Design and build internal platforms, frameworks, and tooling that enable reliable operations at scale.
- Lead complex production incident response and drive systemic improvements through root cause analysis and long-term remediation.
- Partner with engineering leadership to influence platform architecture, deployment strategy, and production readiness.
- Mentor engineers and raise the engineering bar through technical leadership, design reviews, and operational guidelines.
- Collaborate with customers and internal teams to build secure, scalable, and highly reliable deployment architectures for both cloud and on-prem environments.
Benefits
- medical, dental and vision insurance
- a 401(k) plan with a Cisco matching contribution
- paid parental leave
- short and long-term disability coverage
- basic life insurance
- 10 paid holidays per full calendar year, plus 1 floating holiday for non-exempt employees
- 1 paid day off for employee’s birthday, paid year-end holiday shutdown, and 4 paid days off for personal wellness determined by Cisco
- 16 days of paid vacation time per full calendar year, accrued at rate of 4.92 hours per pay period for full-time employees (non-exempt)
- flexible vacation time off program (exempt)
- 80 hours of sick time off provided on hire date and each January 1st thereafter, and up to 80 hours of unused sick time carried forward from one calendar year to the next
- Optional 10 paid days per full calendar year to volunteer
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
