Staff Software Engineer, Applied AI

Okta • San Francisco, CA

7d

About The Position

Okta is building Digital Workers, a suite of autonomous agents for identity operations and security. The team is bringing these agents to Okta customers and is seeking a Staff Software Engineer to build and improve the harnesses and evaluations that make them reliable. The agents investigate risk, use tools, wait for approvals, and complete work across enterprise systems. The role offers ownership of a domain such as agent harnesses, evaluations, or model quality, with responsibility for implementing critical components across workers.

The engineer will develop agent execution capabilities, integrate model APIs, select models based on measured quality, latency, and cost, and build evidence-based evaluations and regression tests. The work also involves connecting agent execution to AWS workflows and approval state, enforcing application policy checks for authorization and sensitive writes, protecting tenant boundaries, and improving prompts, retrieval, tools, and model routing. Okta’s core stack includes Python FastAPI, AWS Step Functions, Amazon Bedrock AgentCore, PostgreSQL on Amazon RDS, and pgvector, with OpenAI and Anthropic APIs planned. The team also plans to add an agent harness and a dedicated analytics layer using ClickHouse or BigQuery; the harness choice is open.

The position is based on building production AI software customers can trust. Candidates should bring experience with tool-using LLM applications and be able to explain their working code and failure analysis. Foundation-model research experience is optional.

Requirements

• Strong Python engineering skills.
• Experience shipping an LLM application that uses tools.
• Hands-on experience building agent workflows or evaluation systems for tool-using LLM applications.
• Experience integrating model APIs, structured outputs, and asynchronous services.
• Depth in agent execution or evaluation systems, with working code and failure analysis the candidate can explain.
• Ability to distinguish a plausible model response from a supported finding or verified action.

Nice To Haves

• Experience with Claude Agent SDK, OpenAI Codex SDK, Strands Agents, LangChain Deep Agents, LangGraph, or a custom agent harness implementation; equivalent tools also count.
• Experience with OpenAI or Anthropic is especially relevant.
• Open-source contributions.
• Experience with agent observability, security evaluations, or synthetic evaluation data.
• Experience with fine-tuning, preference data, post-training evaluation, or model distillation using approved datasets.
• Experience with identity and security workflows.
• Foundation-model research experience is optional.

Responsibilities

• Build an agent harness supporting planning, tool use, memory, context compaction, interruption, and recovery during long-running work.
• Integrate OpenAI and Anthropic model APIs, and select models using measured quality, latency, and cost with clear failure handling.
• Evaluate existing harnesses and build only missing capabilities.
• Connect the harness to AWS workflow execution and approval state.
• Keep authorization and sensitive writes behind application policy checks.
• Require evidence of resulting state before marking work complete.
• Build evaluation datasets and replay tests for valid findings, access decisions, app setup, and AI agent onboarding.
• Measure false positives, missed risks, unsafe actions, completion rate, and cost.
• Turn production failures into regression tests.
• Test prompt injection, stale evidence, conflicting instructions, and unsafe tool use.
• Protect tenant boundaries in retrieval and memory.
• Improve prompts, retrieval, tools, and model routing.
• Evaluate fine-tuning or other post-training methods when results justify them and data use is approved.
• Lead a domain such as harnesses, evaluations, or model quality across workers and implement its critical components.

Benefits

• Equity, where applicable.
• Bonus.
• Health, dental, and vision insurance.
• 401(k).
• Flexible spending account.
• Paid leave, including PTO and parental leave, in accordance with applicable plans and policies.
• In-person onboarding experience.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed