Staff Production Engineer

Zscaler•San Jose, CA

9d•$133,000 - $190,000•Hybrid

About The Position

Zscaler is seeking a Staff Production Engineer to help improve the reliability of its global cloud security platform. This hybrid role requires working three days per week in San Jose, California, or Bellevue, Washington, and reports to the Senior Manager, Site Reliability Engineering, in the Cloud Infrastructure & Operations department. The engineer will provide technical vision and hands-on execution to promote an automation-first culture, mature observability and architectural standards, reduce Mean Time to Mitigate, and shape the scalability of globally distributed, multi-cloud infrastructure. The role covers highly available infrastructure across public clouds and bare-metal environments, reliability engineering, incident leadership, and collaboration with Engineering and partner teams.

Requirements

Foundational understanding of AI/ML technologies and experience leveraging, securing, or positioning AI-driven solutions to optimize outcomes in the functional domain.
At least 8 years of experience managing reliability, scalability, and availability for large-scale production services.
Deep programming expertise, such as Python, Go, or C/C++.
Strong background in networking protocols, Linux/FreeBSD systems, and distributed architecture.
Experience with high-stakes incident management and participation in a 24/7 on-call rotation.
Proficiency in ITIL frameworks and use of incident data to drive service maturity through systematic problem management and technical operability reviews.

Nice To Haves

Proven ability to use AI technologies and workflows to drive measurable operational efficiency.
Experience implementing AI-driven AIOps tools, machine-learning models for anomaly detection, or predictive capacity planning to improve infrastructure reliability and automate incident response.
Extensive experience with AWS, Azure, or GCP; Infrastructure-as-Code tools such as Ansible or Terraform; and large-scale chaos engineering or disaster recovery planning.
Expertise in global routing (BGP), traffic tunneling (GRE, IPSec), L7 proxy architectures (HAProxy), DNS at scale, and OS networking stack internals.

Responsibilities

Design and implement highly available, scalable infrastructure across AWS, Azure, GCP, and bare-metal environments.
Write code in Python or Go to eliminate manual toil and build self-healing systems, advancing an automation-first culture.
Implement and maintain observability using Prometheus, Grafana, and OpenTelemetry; define SLIs and SLOs and establish error budgets.
Lead incidents as Incident Commander in the TDO on-call rotation, develop response playbooks, and conduct in-depth post-incident analyses.
Partner with Engineering and partner teams to conduct operability reviews.