Staff Production Engineer

Zscaler•San Jose, CA

1d•$133,000 - $190,000•Hybrid

About The Position

Zscaler is seeking a Staff Production Engineer to help improve the reliability of its global cloud security platform. The role provides technical vision and hands-on execution to advance an automation-first culture, mature observability and architectural standards, reduce Mean Time to Mitigate, and shape the scalability of globally distributed, multi-cloud infrastructure. This is a hybrid role requiring three days per week in San Jose, California, or Bellevue, Washington, reporting to the Director, Software Engineering in the Cloud Infrastructure & Operations department. The engineer will design and implement highly available infrastructure across public cloud and bare-metal environments; automate operational work and build self-healing systems; implement observability, service-level objectives, and error budgets; lead incident response and post-incident analysis; and partner with engineering teams on operability reviews. The role includes participation in a 24/7 on-call rotation. The successful candidate is comfortable working through ambiguity, takes ownership from strategy through hands-on execution, solves difficult technical problems, collaborates with candor and respect, and actively seeks feedback and development. Zscaler offers a base salary range of $133,000–$190,000 USD for this full-time position, excluding applicable commission, bonus, equity, and benefits. Benefits include health plans, vacation and sick time, parental leave, retirement options, education reimbursement, and in-office perks.

Requirements

Foundational understanding of AI/ML technologies and experience leveraging, securing, or positioning AI-driven solutions to optimize outcomes within the functional domain.
At least 8 years of experience managing reliability, scalability, and availability for large-scale production services, with deep programming expertise in Python, Go, C/C++, or similar.
Strong background in networking protocols, Linux or FreeBSD systems, and distributed architecture.
Experience in high-stakes incident management and participation in a 24/7 on-call rotation.
Proficiency in ITIL frameworks and using incident data to drive service maturity through systematic problem management and technical operability reviews.

Nice To Haves

Ability to leverage AI technologies and workflows to drive measurable operational efficiency.
Experience implementing AI-driven AIOps tools, machine-learning models for anomaly detection, or predictive capacity planning to improve infrastructure reliability and automate incident response.
Extensive public-cloud experience with AWS, Azure, or GCP; Infrastructure-as-Code experience with Ansible or Terraform; and experience with chaos engineering or disaster recovery planning at scale.
Expertise in global routing (BGP), traffic tunneling (GRE, IPSec), L7 proxy architectures (HAProxy), DNS at scale, and OS networking stack internals.

Responsibilities

Design and implement highly available, scalable infrastructure across AWS, Azure, GCP, and bare-metal environments.
Promote an automation-first culture by writing Python or Go code to eliminate manual toil and build self-healing systems.
Implement and maintain observability using Prometheus, Grafana, and OpenTelemetry; define SLIs and SLOs and establish error budgets.
Serve as a lead Incident Commander for TDO on-call, develop response playbooks, and conduct in-depth post-incident analyses.
Partner with Engineering and partner teams to conduct operability reviews.
Manage reliability, scalability, and availability for large-scale production services.
Use incident data and ITIL frameworks to drive service maturity through systematic problem management and technical operability reviews.