Staff AI Engineer, Agentic Automation

Cerby

About The Position

Cerby is hiring a Staff AI Engineer to lead agentic automation for onboarding applications and repairing automations when vendor websites or APIs change. The role begins with two existing, benchmarked systems: one that reads public API documentation in a headless browser and generates a validated OpenAPI specification, and another that performs tasks in SaaS web interfaces and produces replayable automation workflows, including handling MFA, one-time codes, and CAPTCHAs. The engineer will take these systems to production, make them self-healing, and set the direction for agentic automation at Cerby. The systems handle customer credentials and browse untrusted sites, so security is central to the role. The engineer will own architecture, roadmap, production operations, evaluation, cost and latency, and threat modeling; collaborate with integrations, automation-runtime, and platform teams; write design documents; and mentor contributors. Success means both systems operate in production for real workloads within six months, with improving error rates and cost per output and automatic drift detection. Within 12 months, application onboarding is predominantly automated and most drift-related breakages are repaired without human intervention.

Requirements

Have shipped an LLM agent with tool use that ran unattended for real users.
Have owned a backend system in production.
Strong skills in Python or TypeScript and ability to work productively in both.
Experience with LLM agents and tool use, including agent loops, structured outputs, retries, and their failure modes.
Experience building benchmarks or evaluation harnesses and using them to make decisions.
Depth in browser automation such as Playwright, or backend and API-integration systems, and comfort with the other area.
Knowledge of distributed systems, including queues, idempotency, and durable workflow engines such as Temporal or Step Functions.
Sound security judgment for systems handling secrets, including agent-specific threats such as prompt injection.
Clear technical writing and ability to communicate trade-offs other teams can rely on.
Ability to take an early system to production independently and ensure it continues working after launch.

Nice To Haves

Production experience with Temporal.
Experience with Pydantic AI or a comparable agent framework.
Open-weight model serving experience, including Fireworks, Together, or Bedrock, and LLM cost modeling.
Experience with computer-use agents or browser extensions such as Chrome MV3.
OpenAPI tooling experience.
Identity, IAM, or credential-management experience.
AWS and Terraform experience.

Responsibilities

Own the architecture, roadmap, and on-call operations for agentic application onboarding and self-healing automation.
Move existing agent systems to production using durable workflows such as Temporal and establish the observability needed to operate them.
Detect automation drift, including changes to login pages, interface elements, and API fields, from failed runs before customers are affected.
Build agents that diagnose failures, repair workflows or specifications, and revalidate fixes against real runs before returning them to service.
Determine which repairs can occur automatically and which require human review.
Own benchmarks for generated specifications, workflows, and repairs, and establish measured quality thresholds for release.
Manage cost and latency per output, selecting frontier or open-weight models based on measured accuracy and cost.
Own the threat model, including prompt injection, credential isolation, token handling, sandboxed browsing, and egress.
Ensure credentials are excluded from prompts, logs, queues, and stored run state.
Partner with integrations, automation-runtime, and platform teams; write design documents and mentor contributors.