Senior Software Engineer, Agentic Platform - Moveworks

ServiceNow•Mountain View, CA

16h

About The Position

Moveworks is building the runtime infrastructure that powers its AI agents, enabling them to orchestrate and execute workflows and deliver responses to enterprise users in real time. This role combines large language models, machine learning, and distributed systems to make agents plan, reason, and use tools reliably, while building production systems that operate at enterprise scale. The engineer will build and own systems that support long-running agent sessions, multi-step workflows, tool calls, human interaction, and resumable execution while maintaining correctness, observability, and low latency. The work includes agent orchestration, model routing and adaptation, distributed session management, event-driven messaging, structured concurrency, observability, caching and state management, LLM tool connectivity, token efficiency, and evaluation of open-weights models.

Requirements

At least 5 years of experience building production backend or infrastructure systems.
Strong programming skills in Python or Go; experience with both is ideal.
Experience designing and operating systems that handle real traffic at scale.
Hands-on experience building production systems that use LLMs or other machine learning models.
Deep experience in at least three of the following areas: distributed systems; concurrent or asynchronous programming; event-driven architectures; database systems for infrastructure; observability; gRPC/protobuf; or machine learning, including LLMs and NLP.
Comfort working with ambiguity and novel problems without textbook solutions.

Nice To Haves

Experience building and shipping products using LLM techniques such as RAG, chain-of-thought, self-reflection, tool and function calling, and agentic reasoning.
Experience training, fine-tuning, or evaluating machine learning models on large datasets in production, particularly for NLP, retrieval, or ranking.
Experience improving relevance or quality through experimentation, including designing metrics, building evaluation datasets, and developing features or signals for ranking or model selection.
Research or applied experience in machine learning, LLMs, agentic systems, information retrieval, or ranking and relevance.

Responsibilities

Build an agent orchestration engine that manages long-running sessions and coordinates planning, execution, and user interaction across LLM calls and tool invocations.
Select and route between models, and fine-tune or distill LLMs for agent tasks while addressing enterprise requirements such as granular permissions, data freshness, explainability, and preserving general knowledge.
Build distributed session management using lease-based ownership, conditional writes, heartbeat protocols, and checkpoint-based crash recovery.
Develop event-driven message pipelines, including FIFO queues, event consumers, and real-time streaming with gRPC and Socket.IO.
Implement structured concurrency for concurrent session tasks, with fail-fast semantics and graceful cancellation.
Build observability infrastructure using OpenTelemetry, distributed trace context propagation across asynchronous boundaries, and custom span lifecycle management for long-running sessions.
Develop caching and state layers using Redis and DynamoDB, including per-organization and per-bot scoping, batch-read optimization, and hot-reload configuration.
Build MCP and agent-to-agent capabilities, including tool discovery, namespacing, and tool selection across large tool catalogs.
Improve agent token efficiency, cost, and latency through prompt and context discipline, caching, speculative and parallel model-call scheduling, and reducing token use across multi-step reasoning.
Adopt and evaluate open-weights models by deploying newly released models at different quantization levels, benchmarking accuracy, latency, and cost on agent tasks, and determining which models are production-ready.