Senior Staff Software Engineer, Agentic Systems - Moveworks

ServiceNow • Mountain View, CA

2d • Remote

About The Position

We're building the runtime infrastructure that powers Moveworks' AI agents — the systems that orchestrate, execute, and deliver agent responses to millions of enterprise users in real time. This is a distributed systems engineering role at the heart of the agentic AI wave. Our AI agents can plan, execute multi-step workflows, call tools, wait on human input, and resume — all while maintaining correctness, observability, and low latency. The systems that make this possible are what you'll build and own.

Requirements

• Deep experience in at least 3 of the following areas: Distributed systems (consistency models, idempotency, exactly-once delivery, distributed locking/leasing), Concurrent/async programming (Python asyncio, Go goroutines, structured concurrency, cancellation handling), Event-driven architectures (message queues like SQS, Kafka, pub/sub, backpressure, delivery guarantees), Database systems for infrastructure (DynamoDB, Redis), Observability (OpenTelemetry, distributed tracing, span context propagation, Prometheus metrics), gRPC/protobuf (streaming RPCs, service interface design, error handling patterns).
• 10+ years building production backend/infrastructure systems.
• Strong in Python or Go (ideally both).
• Experience designing and operating systems that handle real traffic at scale.
• Comfort with ambiguity — these are novel problems without textbook solutions.

Responsibilities

• Build and own the agent orchestration engine, a state machine that manages long-running agent sessions, coordinating planning, execution, and user interaction across multiple LLM calls and tool invocations.
• Develop distributed session management using lease-based ownership with DynamoDB conditional writes, heartbeat protocols, and crash recovery via checkpointing.
• Implement an event-driven message pipeline using SQS FIFO queues for ordered delivery, Kafka consumers for event processing, and real-time streaming via gRPC and Socket.IO.
• Utilize structured concurrency with Python asyncio TaskGroups for running multiple concurrent tasks per session (message polling, lease heartbeats, output publishing, orchestrator execution) with fail-fast semantics and graceful cancellation.
• Enhance observability infrastructure with OpenTelemetry instrumentation, distributed trace context propagation across async boundaries, and custom span lifecycle management for sessions that span minutes.
• Develop caching and state layers using Redis and DynamoDB KV stores with per-org/per-bot scoping, batch read optimization, and hot-reload configuration.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed