Staff Site Reliability Engineer, Federal (TS/SCI)

Okta•Washington, DC

10d•$174,000 - $238,000•Hybrid

About The Position

This role is ideal for an engineer who enjoys solving complex technical challenges at scale, building automation, and improving the reliability of production systems. You will serve as a technical leader within the EPG SRE organization, partnering closely with software engineers, architects, and product teams to design, build, and operate world-class cloud services. The ideal candidate exemplifies the philosophy of "if you have to do it more than once, automate it" and possesses a strong passion for continuous improvement, operational excellence, and software engineering. This position requires an active U.S. TS/SCI clearance with Full Scope Poly Compliance and proven experience navigating Federal and DoD compliance frameworks, specifically FedRAMP and Impact Level 6 (IL6).

Requirements

Active U.S. TS/SCI clearance with Full Scope Poly Compliance
Proven experience navigating Federal and DoD compliance frameworks, specifically FedRAMP and Impact Level 6 (IL6)
Strong experience operating large-scale production services in AWS and/or GCP.
Deep expertise with Kubernetes in production environments.
Experience troubleshooting Kubernetes networking, storage, scheduling, scaling, and workload lifecycle issues.
Extensive experience with Infrastructure as Code technologies such as Terraform and Helm.
Strong software engineering skills in Golang and/or Python.
Experience building automation and internal engineering platforms.
Experience operating and troubleshooting distributed data platforms such as PostgreSQL, Redis, OpenSearch, MySQL, Cassandra, or similar technologies.
Strong understanding of cloud networking fundamentals including DNS, load balancing, ingress, TLS, service networking, and traffic management.
Experience with observability platforms, monitoring strategies, and production telemetry.
Experience with or strong interest in AI-assisted engineering and operational automation.
Strong expertise operating customer-facing production systems.
Experience leading incident response and driving operational improvements.
Deep understanding of reliability engineering concepts including SLIs, SLOs, error budgets, and capacity planning.
Strong understanding of CI/CD pipelines, deployment strategies, and automation-first operational practices.
Proven ability to balance reliability, scalability, security, and engineering velocity.
Understanding of cloud security fundamentals, IAM, secrets management, and secure infrastructure design.
Demonstrated success leading complex engineering initiatives across multiple teams.
Strong collaboration and communication skills.
Experience working effectively within globally distributed engineering organizations spanning multiple timezones and cultures.
Experience mentoring engineers and elevating technical capabilities within an organization.
Ability to influence technical direction through expertise, partnership, and execution.
U.S. soil status - the employee must be on U.S. soil, which means the 50 states, the District of Columbia, or outlying areas of the United States, as defined in Federal Acquisition Regulation (FAR) 2.101
U.S. Security Clearance status - the employee must be able to obtain and maintain a U.S. security clearance (Secret or Top Secret) to the extent required by U.S. Government contracts.

Nice To Haves

Experience implementing operational controls and best practices in regulated or security-sensitive environments is a plus.
Experience operating SaaS platforms serving large-scale customer workloads.
Experience working within Kubernetes-based microservices environments.
Experience supporting globally distributed production environments.
Experience with GitOps and ArgoCD.
Experience implementing AI-assisted operational tooling or automation workflows.

Responsibilities

Design, build, and operate large-scale cloud infrastructure and production services.
Participate in an on-call rotation supporting highly available customer-facing systems.
Lead incident response efforts and drive post-incident reviews focused on systemic improvements.
Define, measure, and improve Service Level Indicators (SLIs), Service Level Objectives (SLOs), and error budgets.
Partner with engineering teams to improve service availability, scalability, performance, and resilience.
Continuously improve observability through metrics, logging, tracing, dashboards, and alerting.
Develop software, automation, and infrastructure using Go, Python, Terraform, and related technologies.
Eliminate operational toil through automation, tooling, and platform engineering.
Improve deployment safety and operational workflows through CI/CD and GitOps practices.
Collaborate on modernizing existing workloads and aligning them with evolving platform capabilities.
Build self-service platforms, operational guardrails, and automation that improve developer velocity while maintaining reliability and security.
Lead complex reliability initiatives spanning multiple engineering teams.
Guide engineers in adopting operational best practices and reliability engineering principles.
Mentor engineers through technical collaboration, design reviews, incident analysis, and knowledge sharing.
Influence architecture and operational decisions through data-driven recommendations and engineering expertise.
Drive projects from conception through production rollout and long-term operational ownership.
Explore and apply AI-assisted engineering techniques to improve operational efficiency, incident response, troubleshooting, and automation.
Identify opportunities to leverage emerging technologies to reduce toil and improve engineering productivity.