Staff Machine Learning Engineer

ServiceNow•Santa Clara, CA

2d•$176,100 - $308,200

About The Position

The Staff AI Engineer will own significant parts of the agent harness, the infrastructure layer that enables AI agents to reason over enterprise data, take action across workflows, and operate safely at Fortune 500 scale. The role is part of ServiceNow’s Agentic Engineering organization and specifically the Agent Orchestration team, which builds the execution core for conversational AI, including the agent harness, orchestration runtime, multi-agent coordination, memory management, and production evaluation frameworks. The engineer will develop reliable, observable, and efficient agent execution infrastructure; build prompt management and evaluation systems; integrate and abstract over frontier LLMs; and help establish technical standards for agentic design and production AI. The role also provides technical leadership through architecture decisions, code reviews, and coaching. The position offers a base pay range of $176,100–$308,200, plus equity when applicable, variable or incentive compensation, and benefits. Compensation varies according to qualifications, skills, competencies, and work location. The role is designated as a flexible work persona; eligibility and arrangements depend on the assigned work location and the nature of the work.

Requirements

At least 7 years building production software systems, with a strong track record in reliability, performance, and scalability.
Hands-on experience shipping generative AI products, including ownership of AI-powered features used by production users.
Solid understanding of how large language models work, including failure modes, context constraints, and how prompt design affects behavior at scale.
Practical prompt engineering experience, including systematically designing, versioning, and evaluating prompts across model updates or A/B evaluation cycles.
A track record in evaluation engineering, including designing, shipping, and using evaluation suites to drive quality decisions in production AI systems.
System-level cost and efficiency awareness, including experience reasoning about model routing, inference cost, and latency trade-offs in production.
Strong software engineering fundamentals, including distributed systems, API design, and testing discipline.
Comfort working in fast-moving, ambiguous, startup-like AI product environments.

Nice To Haves

Experience with multi-agent coordination patterns such as A2A or MCP.
Familiarity with agent frameworks such as LangChain, LlamaIndex, or similar.
Prior experience shipping AI systems in enterprise software.
Experience with AI observability tooling, including tracing, cost tracking, or LLM-specific monitoring.
Familiarity with cloud-native infrastructure, service observability, logging, monitoring, reliability engineering, and production troubleshooting.

Responsibilities

Design and build the agent execution harness to route inputs, manage context, invoke tools, handle retries, and surface execution state across multi-step agentic workflows.
Own runtime fault tolerance, latency, and throughput for enterprise workflows.
Instrument the harness with tracing, cost attribution, and latency visibility to help assess agent behavior in production and detect failures.
Build prompt management systems for versioning, templating, and systematic evaluation to maintain stable agent behavior across model and configuration changes.
Design and own evaluation frameworks, including unit evaluations, integration evaluations, and production monitors, to measure agent quality, catch regressions, and inform decisions.
Integrate with and abstract over frontier LLMs, managing model routing, fallback strategies, cost, and latency trade-offs.
Set technical standards through architecture decisions, code reviews, and coaching, especially for agentic design patterns and production AI practices.
Define system boundaries for agent logic, including decisions about tool calls, sub-agents, hardcoded paths, and human escalation, and establish design standards across the team.