Staff Software Engineer, AI Platform & Infrastructure

Okta•San Francisco, CA

12h•$194,000 - $267,300

About The Position

The Staff Software Engineer will help build Digital Workers, a suite of autonomous agents for identity operations and security at Okta. The role focuses on backend services and AWS platform infrastructure that keep the workers running continuously. The team owns services, connectors, workflow execution, data, and production reliability. The engineer will build Python services and infrastructure that allow workers to make approved changes while preserving evidence, with responsibility for deployment, monitoring, and production support. At the Staff level, the engineer leads a domain such as execution, tenant security, data infrastructure, or reliability across teams. The team operates with startup-style ownership and pace while drawing on the resources of a public company. Its product is intended to reach Okta's 20,000+ customers. The role uses Python FastAPI and AWS services, with data and security systems including RDS PostgreSQL, pgvector, S3 Object Lock, KMS, Secrets Manager, CloudWatch, and CloudTrail. React and AWS Amplify are used for the frontend. OpenAI and Anthropic integrations and dedicated analytics are planned; analytics technology choices remain open.

Requirements

Experience building and operating production backend or distributed systems, with strong coding skills in Python or a comparable language.
Hands-on AWS experience with APIs, queues, containers, networking, IAM, and databases.
Strong SQL and data modeling skills.
Experience diagnosing database performance and asynchronous system failures.
Experience with infrastructure as code, deployment automation, observability, and production incident response.
Sound judgment about customer data, tenant isolation, credentials, and safe changes to external systems.

Nice To Haves

Experience with ClickHouse, BigQuery, or comparable analytics systems, event ingestion, and usage metering.
Experience with multi-tenant SaaS, OAuth, OIDC, SAML, SCIM, or enterprise connector development.
Experience deploying agent frameworks, including Claude Agent SDK, Codex SDK, Strands Agents, Deep Agents, or LangGraph.
Identity experience is optional.

Responsibilities

Build Python FastAPI services, data models, and reusable connectors for Okta, AWS, ServiceNow, and enterprise applications.
Implement durable workflow execution using Step Functions, SQS, and EventBridge, including approval waits, rate limits, duplicate events, and safe retries.
Manage PostgreSQL schemas, indexes, query performance, and pgvector retrieval; enforce tenant isolation across APIs, storage, and credentials.
Build model API access for OpenAI, Anthropic, and supported AWS paths, including credentials, quotas, timeouts, and cost controls.
Create an analytics pipeline for worker events, traces, tokens, findings, and verified outcomes; evaluate ClickHouse and comparable systems such as BigQuery.
Build reliable usage records and reconciliation, preventing retries, duplicate findings, and incomplete work from being counted as new outcomes.
Build deployment pipelines, infrastructure as code, observability, and safe release controls across development, staging, and production.
Implement scoped write services, kill switches, audit retention, and backup recovery; test failures and verify that sensitive changes took effect.
Lead a domain such as execution, tenant security, data infrastructure, or reliability across teams.