Senior Software Engineer, Agentic Platform - Moveworks

ServiceNow•Mountain View, CA

1d•Onsite

About The Position

The Senior Software Engineer, Agentic Platform will build and own the runtime infrastructure that powers Moveworks’ AI agents. The role combines large language models, machine learning, and distributed systems to help agents plan, reason, use tools reliably, and deliver responses to enterprise users in real time. The systems must support multi-step workflows, human input, resumable sessions, correctness, observability, and low latency at enterprise scale. The engineer will develop an agent orchestration engine and model routing and adaptation capabilities, as well as distributed session management, event-driven messaging, structured concurrency, observability, and caching and state layers. The role also includes building MCP and agent-to-agent capabilities for tool discovery and selection, improving token efficiency and context engineering, and evaluating newly released open-weights models for potential production use.

Requirements

At least 5 years of experience building production backend or infrastructure systems.
Strong proficiency in Python or Go; ideally both.
Experience designing and operating systems that handle real traffic at scale.
Hands-on experience building production systems that use LLMs or other machine-learning models.
Deep experience in at least 3 of the following areas: distributed systems; concurrent or asynchronous programming; event-driven architectures; database systems for infrastructure; observability; gRPC/protobuf; machine learning, LLMs, or natural language processing.
Comfort working with ambiguity and novel problems without textbook solutions.

Nice To Haves

Experience building and shipping products using LLM techniques such as RAG, chain-of-thought, self-reflection, tool and function calling, and agentic reasoning.
Experience training, fine-tuning, or evaluating machine-learning models on large datasets in production, especially for NLP, retrieval, or ranking.
Experience improving relevance or quality through experimentation, metric design, evaluation datasets, and new ranking or model-selection features and signals.
Research or applied experience in machine learning, LLMs, agentic systems, information retrieval, or ranking and relevance.

Responsibilities

Build and own runtime infrastructure that orchestrates, executes, and delivers AI agent responses at enterprise scale.
Develop an agent orchestration state machine for long-running sessions spanning planning, execution, user interaction, LLM calls, and tool invocations.
Choose, route, fine-tune, or distill LLMs for agent tasks while addressing enterprise requirements such as permissions, data freshness, explainability, and preserving general knowledge.
Implement distributed session management using lease-based ownership, conditional writes, heartbeat protocols, and checkpoint-based crash recovery.
Build event-driven message pipelines with ordered FIFO delivery, event-processing consumers, and real-time streaming over gRPC and Socket.IO.
Implement structured concurrency for concurrent session tasks, including fail-fast behavior and graceful cancellation.
Build observability infrastructure using OpenTelemetry, distributed trace context propagation, and custom span lifecycle management for long-running sessions.
Build and optimize caching and state layers using Redis and DynamoDB, including organization- and bot-scoped storage, batch reads, and hot-reload configuration.
Build MCP and agent-to-agent capabilities, including tool discovery, namespacing, and tool selection across large tool catalogs.
Improve agent token efficiency, cost, and latency through prompt and context discipline, caching, speculative and parallel model-call scheduling, and reducing tokens spent on reasoning chains.
Set up newly released open-weights LLMs at different quantization levels, benchmark them on agent tasks for accuracy, latency, and cost, and determine which models are suitable for production.
Design, build, and operate production backend and infrastructure systems that handle real traffic at scale.
Develop and evaluate LLM- or ML-based products and features, using experimentation and metrics to improve relevance and quality.