Principal Software Development Engineer- Go

Zscaler•Santa Clara, CA

1d•$185,000 - $265,000•Hybrid

About The Position

Zscaler is seeking a Principal Software Development Engineer for its Service Platform & Orchestration team in the Zero Trust Exchange department. This is a hybrid role, with three days in the office, reporting to the Director, Software Engineering. The engineer will develop the management plane and reliability systems that govern the ZIA fleet lifecycle at massive scale. This hands-on role will lead the transformation of infrastructure from legacy automation to a stateful, durable management plane built on Temporal. The goal is deterministic, one-touch provisioning and lifecycle operations, treating infrastructure as a distributed system and developing self-healing capabilities and AI-driven SRE practices for a global fleet of more than 100,000 instances. Zscaler’s Zero Trust Exchange platform securely connects users, devices, and applications and protects customers from cyberattacks and data loss. The company describes its future of work as Human + AI and aims to use machine intelligence to address security challenges.

Requirements

Foundational understanding of AI/ML technologies and experience leveraging, securing, or positioning AI-driven solutions to optimize outcomes in the candidate’s functional domain.
BS/MS in Computer Science or a related technical field, with 12+ years of experience in hyperscale systems.
Deep understanding of failure modes and technical challenges that emerge at massive scale.
Mastery of backend systems languages, with a strong preference for Go; alternatively, deep expertise in Java, C++, or Python and demonstrated ability to rapidly adopt Go.
Proven record of setting a high bar for code quality and distributed-system correctness.
Strong experience designing and operating complex distributed systems, particularly addressing concurrency, failure handling, and performance optimization.
Proven track record developing Platform APIs (REST/gRPC) with strong guarantees for idempotency, verification, and safe rollout patterns.

Nice To Haves

Proficiency with AI code-assistance tools such as Cursor or Windsurf for legacy refactoring and system development.
Proficiency in PostgreSQL or other relational stores used for high-scale, stateful management-plane services.
Direct experience building or operating systems with Temporal.io, Cadence, or similar workflow engines.

Responsibilities

Lead hands-on development and migration to a workflow-as-code platform using Temporal, building replay-safe, idempotent workflows for deterministic operations at global scale.
Build a robust management plane that treats the global fleet as a single, eventually consistent distributed system rather than relying on scripted automation.
Design and implement services using LLMs and ML for signal correlation, automated triage, and self-correcting fleet operations.
Develop framework-level services and internal APIs so new products are orchestration-ready, with reliability hooks built into the code.
Build metrics, traces, and events into the management plane so fleet-wide actions are explainable, auditable, and replayable.