Sr Principal DevOps Engineer (AIOps)
About The Position
Palo Alto Networks is hiring a Sr Principal DevOps Engineer to support services running on its large hybrid infrastructure, including infrastructure that uses GCP and AWS. The role encompasses automation, architecture, performance, metrics, troubleshooting, security, and reliability. The technology stack includes Kubernetes, Docker, GCP, AWS, Ansible, Terraform, Vault, GitLab, Spinnaker, TensorFlow, Datadog, Elasticsearch, Kafka, Hadoop, MySQL, Percona, MongoDB, Python, and Go; candidates are not expected to know every technology but should be prepared to learn those needed for the role. The engineer will shape organization-level SRE and platform engineering strategy, architect highly available multi-region and multi-cloud systems, establish reliability standards and governance, and lead platform abstraction and observability efforts. The role also includes production readiness and optimization, major incident leadership, influence across product and engineering teams, mentoring, and driving automation and operational excellence.
Requirements
- 12–15+ years of experience in infrastructure, SRE, DevOps, or systems engineering.
- 8–10+ years building and operating large-scale distributed systems on AWS and/or GCP.
- Proven experience driving cross-team or organization-level technical initiatives.
- Strong expertise in Terraform, Helm, or similar infrastructure-as-code frameworks.
- Deep experience in automation using Python, Go, or shell scripting.
- Strong hands-on expertise in Kubernetes at scale.
- Advanced knowledge of Linux, networking, and distributed-systems debugging.
- Experience with CI/CD using GitLab and GitOps using ArgoCD.
- Strong background in observability, including Prometheus, Grafana, Loki, and tracing.
- Ability to lead incident management and complex production troubleshooting.
- Excellent communication skills and ability to influence directors and VPs.
- Demonstrated leadership in mentoring and setting engineering standards.
- Strong ownership, with a focus on scalability, reliability, and long-term architecture.
Responsibilities
- Define and drive organization-level SRE and platform engineering strategy.
- Architect highly available, multi-region, multi-cloud systems across GCP and AWS.
- Establish SLO/SLI standards, error budgets, and reliability governance across teams.
- Lead the design of platform abstractions, including Terraform modules, Kubernetes platforms, and GitOps.
- Drive end-to-end observability architecture and reduce mean time to recovery at scale.
- Ensure services are production-ready, scalable, secure, and cost-optimized.
- Lead SEV-1 incidents, root-cause analysis, and executive-level communication.
- Influence product and engineering teams on architecture and reliability decisions.
- Mentor senior engineers and act as a technical authority across organizations.
- Drive automation, self-service platforms, and operational excellence.
Benefits
- Compensation may include restricted stock units and a bonus.
- Employee benefits are referenced, but specific benefits are not listed.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Principal
Education Level
No Education Listed
