Staff Machine Learning Engineer

ServiceNow•Santa Clara, CA

17h

About The Position

The Staff AI Engineer will own significant parts of the agent harness, the infrastructure layer that enables AI agents to reason over enterprise data, take action across workflows, and operate safely at Fortune 500 scale. This role is part of ServiceNow’s Agent Orchestration team within the Agentic Engineering organization, which builds the execution core for conversational AI, including orchestration, multi-agent coordination, memory management, and evaluation frameworks. The position combines hands-on engineering and technical leadership. The engineer will build and maintain the agent execution runtime, ensure its reliability and observability, develop prompt and evaluation infrastructure, integrate frontier language models, and set technical standards for agentic systems in production. The role also establishes system boundaries for when to use agent logic, tools, sub-agents, fixed paths, or human escalation.

Requirements

At least 7 years of experience building production software systems, with a strong track record in reliability, performance, and scalability.
Hands-on experience shipping generative AI products and owning AI-powered features relied on by production users.
Strong understanding of how large language models work, including failure modes, context constraints, and the effects of prompt design at scale.
Experience with Python, Kubeflow, and SQL.
Experience with SageMaker, Vertex AI, or AzureML.
Practical prompt engineering experience, including systematically designing, versioning, and evaluating prompts across model updates or A/B evaluation cycles.
A demonstrated track record in evaluation engineering, including designing and shipping evaluation suites used to make production AI quality decisions.
Experience reasoning about model routing, inference cost, and latency trade-offs in production.
Strong software engineering fundamentals, including distributed systems, API design, and testing discipline.
Comfort operating in fast-moving, ambiguous, startup-like AI product environments.

Nice To Haves

Experience with multi-agent coordination patterns such as A2A or MCP.
Familiarity with agent frameworks such as LangChain, LlamaIndex, or similar.
Prior experience shipping AI systems in enterprise software.
Experience with AI observability tooling, including tracing, cost tracking, or LLM-specific monitoring.
Familiarity with cloud-native infrastructure, service observability, logging, monitoring, reliability engineering, and production troubleshooting.

Responsibilities

Design and build the agent execution harness to route inputs, manage context, invoke tools, handle retries, and surface execution state across multi-step workflows.
Own runtime fault tolerance, latency, and throughput for enterprise workflows.
Instrument the harness with tracing, cost attribution, and latency visibility to support production troubleshooting and early failure detection.
Build prompt management systems for versioning, templating, and systematic evaluation across model updates and configuration changes.
Design and own unit evaluation, integration evaluation, and production monitoring frameworks to measure agent quality, catch regressions, and guide decisions.
Integrate and abstract over frontier LLMs, managing model routing, fallback strategies, cost, and latency trade-offs in production.
Set technical standards through architecture decisions, code reviews, and coaching, particularly for agentic design patterns and production AI practices.
Define system boundaries for agent logic, tool calls, sub-agents, hardcoded paths, and human escalations, and establish design standards across the team.