Senior Software Engineer, Agentic Platform - Moveworks

ServiceNow•Mountain View, CA

About The Position

Moveworks is building the runtime infrastructure that powers its AI agents, which orchestrate, execute, and deliver agent responses to enterprise users in real time. This role combines large language models, machine learning, and distributed systems to help agents plan, reason, and use tools reliably, while building production systems that operate at enterprise scale. The engineer will build and own systems that let agents carry out multi-step workflows, call tools, wait for human input, and resume while maintaining correctness, observability, and low latency. The work spans agent orchestration, model routing and adaptation, distributed session management, event-driven messaging, structured concurrency, observability, caching and state, agent and tool connectivity, token efficiency, and evaluation of open-weights models.

Requirements

At least 5 years of experience building production backend or infrastructure systems.
Strong programming skills in Python or Go; ideally both.
Experience designing and operating systems that handle real traffic at scale.
Hands-on experience building production systems that use LLMs or other machine-learning models.
Deep experience in at least three of the following areas: distributed systems; concurrent or asynchronous programming; event-driven architectures; database systems for infrastructure; observability; gRPC/protobuf; or machine learning.
Comfort working with ambiguity and novel problems without textbook solutions.

Nice To Haves

Experience building and shipping products on LLMs using techniques such as RAG, chain-of-thought, self-reflection, tool and function calling, and agentic reasoning.
Experience training, fine-tuning, or evaluating machine-learning models on large datasets in production, especially for NLP, retrieval, or ranking.
Experience improving relevance or quality through experimentation, including designing metrics, building evaluation datasets, and developing ranking or model-selection features and signals.
Research or applied experience in machine learning, LLMs, agentic systems, information retrieval, or ranking and relevance.

Responsibilities

Build an agent orchestration engine that manages long-running sessions and coordinates planning, execution, and user interaction across LLM calls and tool invocations.
Choose and route between models, and fine-tune or distill LLMs for specific agent tasks while addressing enterprise requirements such as granular permissions, data freshness, explainability, and preservation of general knowledge.
Build distributed session management using lease-based ownership, conditional writes, heartbeat protocols, and checkpointing for crash recovery.
Develop event-driven message pipelines with ordered FIFO delivery, event-processing consumers, and real-time streaming using gRPC and Socket.IO.
Implement structured concurrency for multiple concurrent tasks per session, with fail-fast behavior and graceful cancellation.
Build observability infrastructure using OpenTelemetry, distributed trace context propagation across asynchronous boundaries, and custom span lifecycle management for long-running sessions.
Develop caching and state layers using Redis and DynamoDB KV stores, including per-organization and per-bot scoping, batch-read optimization, and hot-reload configuration.
Build MCP and agent-to-agent capabilities for tool discovery, namespacing, and tool selection across large tool catalogs.
Improve token efficiency and agent cost and latency through prompt and context discipline, caching, speculative and parallel model-call scheduling, and reducing tokens across multi-step reasoning chains.
Stand up newly released open-weights LLMs at different quantization levels, test and benchmark their accuracy, latency, and cost on agent tasks, and determine which models are suitable for production.