Staff Production Engineer

Zscaler•San Jose, CA

8d•Hybrid

About The Position

Zscaler is seeking a Staff Production Engineer for its Cloud Infrastructure & Operations department. This hybrid role is based in San Jose, California, or Bellevue, Washington, with three days per week in the office. Reporting to the Senior Manager, Site Reliability Engineering, the role provides technical vision and hands-on execution to improve reliability across Zscaler’s global platform, which protects more than 15 million users. The engineer will advance an automation-first culture, mature observability and architectural standards, reduce Mean Time to Mitigate (MTTM), and help scale globally distributed, multi-cloud infrastructure.

Requirements

Foundational understanding of AI/ML technologies and experience leveraging, securing, or positioning AI-driven solutions to optimize outcomes in the functional domain.
At least 8 years of experience managing reliability, scalability, and availability for large-scale production services.
Deep programming expertise, such as Python, Go, or C/C++.
Strong background in networking protocols, Linux or FreeBSD systems, and distributed architecture.
Experience in high-stakes incident management and participation in a 24/7 on-call rotation.
Proficiency with ITIL frameworks and incident data to improve service maturity through systematic problem management and technical operability reviews.

Nice To Haves

Ability to use AI technologies and workflows to drive measurable operational efficiency.
Experience implementing AI-driven AIOps tools, machine-learning models for anomaly detection, or predictive capacity planning to improve infrastructure reliability and automate incident response.
Extensive experience with public cloud platforms (AWS, Azure, GCP), Infrastructure-as-Code tools (Ansible, Terraform), and large-scale chaos engineering or disaster recovery planning.
Expertise in global routing (BGP), traffic tunneling (GRE, IPSec), L7 proxy architectures (HAProxy), DNS at scale, and OS networking stack internals.

Responsibilities

Design and implement highly available, scalable infrastructure across AWS, Azure, GCP, and bare-metal environments.
Promote an automation-first culture by writing Python or Go code to eliminate manual toil and build self-healing systems.
Implement and maintain observability using Prometheus, Grafana, and OpenTelemetry; define SLIs and SLOs and establish error budgets.
Serve as a lead Incident Commander on the TDO on-call rotation, develop response playbooks, and conduct in-depth post-incident analyses.
Partner with Engineering and other teams to conduct operability reviews.