Sr. Site Reliability Engineer- Federal
About The Position
Zscaler is seeking a Sr. Site Reliability Engineer (Production Engineer) to join the Zero Trust Exchange team. This hybrid position is onsite three days a week in Santa Clara, California, and reports to the Senior Manager, Site Reliability Engineering. The engineer will own systems-level reliability and performance for high-throughput bare-metal and cloud infrastructure processing tens of billions of daily transactions across a global, multi-region fleet. This is a software-first SRE role focused on production-grade code and automation, reducing reactive incident response, and applying engineering discipline to OS, network, and application debugging. The role supports large-scale Linux/BSD fleets, Kubernetes clusters, and custom routing stacks in partnership with Engineering and Networking teams. It includes incident response and cross-stack troubleshooting, infrastructure lifecycle automation, end-to-end telemetry and SLO ownership, architectural reviews, upgrades, capacity and performance tuning, and production rollout validation. The job requires US citizenship due to the nature of assigned customers.
Requirements
- US citizenship is required due to the nature of assigned customers.
- Foundational understanding of AI/ML technologies and experience leveraging, securing, or positioning AI-driven solutions to optimize outcomes within the functional domain.
- At least 3 years of experience in Site Reliability Engineering, Production Engineering, or Systems Engineering operating high-scale, low-latency production platforms.
- Ability to write and debug executable code live in Python, Go, or Bash, covering core logic and data structures.
- Hands-on experience writing Ansible playbooks and tasks for infrastructure automation.
- Deep knowledge of Linux OS internals and kernel troubleshooting, including inodes, open file descriptors, process states, and analyzing df versus du storage discrepancies.
- Comprehensive understanding of networking protocols and packet-level analysis, including DNS resolution workflows, TLS handshakes, TCP/IP mechanics, and packet captures using tcpdump.
Nice To Haves
- Hands-on experience operating and managing FreeBSD/BSD operating systems in production.
- Expertise running, scaling, and troubleshooting Kubernetes clusters in high-traffic, low-latency environments.
- Experience with workflow orchestration platforms such as Temporal or similar.
- Deep experience with Prometheus/OpenTelemetry ecosystems, or use of AI/ML frameworks or AIOps tools for automated root-cause analysis.
Responsibilities
- Maintain high availability across large-scale bare-metal Linux/BSD fleets, Kubernetes clusters, and custom routing stacks in partnership with Engineering and Networking teams.
- Lead full-cycle incident response and cross-stack troubleshooting using low-level operating system and network tools.
- Automate infrastructure lifecycle management, service provisioning, configuration workflows, and release deployments with Ansible, Python, and Go.
- Measure operational toil and convert recurring manual work into durable, version-controlled, testable automation, tracking toil reduction as an engineering outcome.
- Own end-to-end metrics, logs, and traces using Prometheus and OpenTelemetry ecosystems.
- Define and enforce SLOs and error budgets to reduce alert noise.
- Perform architectural reviews, OS/kernel upgrades, capacity planning, and performance tuning.
- Validate production rollouts through strict CI/CD practices and incorporate telemetry, rollback safety, and SLO readiness into service design.
Benefits
- Various health plans
- Time off plans for vacation and sick time
- Parental leave options
- Retirement options
- Education reimbursement
- In-office perks
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
