Senior Applied Research Scientist

ServiceNow

6d

About The Position

The Agentic Engineering organization at ServiceNow is a customer-obsessed engineering group focused on building a conversational AI experience that transforms enterprise intent into completed work. We are advancing how enterprise AI reasons, remembers, and executes. The Agent Orchestration team, which you will join, is responsible for the execution core, including the agent harness, orchestration runtime, multi-agent coordination, memory management, and evaluation frameworks that ensure agents function correctly in production. Every autonomous action promised by Otto depends on the work of this team. Joining us places you at the forefront of our AI transformation, supported by ServiceNow's global scale and the agility of a high-growth environment. We are seeking world-class talent to help us extend agentic AI to every employee across all business sectors.

Requirements

• 4+ years building production software systems with a strong track record on reliability, performance, and scalability.
• Hands-on experience shipping generative AI products, including owning AI-powered features that production users depend on, not just integrating LLM APIs or building prototypes.
• Solid understanding of how large language models work: failure modes, context constraints, and how prompt design influences model behavior at scale.
• Practical prompt engineering experience: systematically designing, versioning, and evaluating prompts across model updates or A/B evaluation cycles.
• A proven track record in evaluation engineering: designing, shipping, and using evaluation suites to drive quality decisions in production AI systems.
• Awareness of cost and efficiency at the system level: experience reasoning about model routing, inference cost, and latency tradeoffs in production.
• Strong software engineering fundamentals: distributed systems, API design, and testing discipline.
• Comfort operating in fast-moving, ambiguous, startup-like AI product environments.

Nice To Haves

• Experience with multi-agent coordination patterns (A2A, MCP).
• Familiarity with agent frameworks (LangChain, LlamaIndex, or similar).
• Prior experience shipping AI systems in enterprise software.
• Experience with AI observability tooling (tracing, cost tracking, LLM-specific monitoring).
• Familiarity with cloud-native infrastructure, service observability, logging, monitoring, reliability engineering, and production troubleshooting.

Responsibilities

• Design and build the agent execution harness, the orchestration layer that routes inputs, manages context, invokes tools, handles retries, and surfaces execution state across multi-step agentic workflows.
• Ensure the runtime's fault tolerance, latency, and throughput; design for enterprise workflows that cannot fail silently or non-deterministically.
• Instrument the harness with tracing, cost attribution, and latency visibility to enable the team to understand agent behavior in production and detect failures before customers do.
• Build prompt management systems, including versioning, templating, and systematic evaluation, to maintain stable agent behavior across model updates and configuration changes.
• Design and own evaluation frameworks (unit evaluations, integration evaluations, production monitors) to measure agent quality, catch regressions, and drive data-informed decisions.
• Integrate with and abstract over frontier LLMs, managing model routing, fallback strategies, cost, and latency tradeoffs in production.
• Elevate the technical standard through architecture decisions, code reviews, and coaching, with a focus on agentic design patterns and production AI discipline.
• Define where agent logic resides (e.g., tool calls, sub-agents, hardcoded paths, human escalations) and establish these design standards across the team.

Benefits

• Flexible work personas (flexible, remote, or required in office).
• Consideration for employment without regard to race, color, religion, sex, sexual orientation, national origin, age, disability, gender identity, veteran status, or any other category protected by law.
• Consideration for employment in accordance with legal requirements for applicants with arrest or conviction records.
• Reasonable accommodation for candidates requiring assistance during the application process.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed