Senior Staff Software Engineer, Agentic Systems - Moveworks

ServiceNow•Mountain View, CA

18h•$201,300 - $352,300•Onsite

About The Position

Moveworks is building runtime infrastructure that powers AI agents, orchestrating, executing, and delivering agent responses to enterprise users in real time. This is a distributed systems engineering role, rather than an ML role. The engineer will build and own systems that enable agents to plan and execute multi-step workflows, call tools, wait for human input, and resume while maintaining correctness, observability, and low latency. The work includes developing an agent orchestration engine; distributed session management with lease-based ownership, heartbeats, and crash recovery; event-driven messaging and real-time streaming; structured concurrency for concurrent session tasks; observability infrastructure; and caching and state layers.

Requirements

At least 10 years building production backend or infrastructure systems.
Strong experience in Python or Go; ideally both.
Experience designing and operating systems that handle real traffic at scale.
Deep experience in at least three of the following areas: distributed systems; concurrent or asynchronous programming; event-driven architectures; database systems for infrastructure; observability; or gRPC/protobuf.
Comfort with ambiguity and novel problems without textbook solutions.

Nice To Haves

Experience with distributed systems, including consistency models, idempotency, exactly-once delivery, and distributed locking or leasing.
Experience with concurrent or asynchronous programming, including Python asyncio, Go goroutines, structured concurrency, and cancellation handling.
Experience with event-driven architectures, including SQS, Kafka, pub/sub, backpressure, and delivery guarantees.
Experience with DynamoDB conditional writes and transactions, and Redis connection pooling or pub/sub.
Experience with OpenTelemetry, distributed tracing, span context propagation, and Prometheus metrics.
Experience with gRPC/protobuf streaming RPCs, service interface design, and error-handling patterns.

Responsibilities

Build and own an agent orchestration engine that manages long-running sessions and coordinates planning, execution, and user interaction across LLM calls and tool invocations.
Develop distributed session management using lease-based ownership, heartbeat protocols, and checkpoint-based crash recovery.
Build event-driven message pipelines using SQS FIFO queues, Kafka consumers, and real-time streaming with gRPC and Socket.IO.
Implement structured concurrency for concurrent per-session tasks, with fail-fast behavior and graceful cancellation.
Develop observability infrastructure with OpenTelemetry, distributed trace context propagation, and session-level span lifecycle management.
Build caching and state layers using Redis and DynamoDB, including per-organization/per-bot scoping, batch-read optimization, and hot-reload configuration.