Staff Machine Learning Engineer

ServiceNow•Santa Clara, CA

1h•$176,100 - $308,200

About The Position

The Agentic Engineering organization builds conversational AI experiences that turn enterprise intent into completed work. The Agent Orchestration team owns the execution core, including the agent harness, orchestration runtime, multi-agent coordination, memory management, and evaluation frameworks that help ensure agents behave correctly in production. The role owns significant parts of the agent harness: infrastructure that enables AI agents to reason over enterprise data, act across workflows, and operate safely at Fortune 500 scale. Responsibilities include designing orchestration capabilities, ensuring reliability and performance, instrumenting production behavior, building prompt management and evaluation systems, integrating large language models, and setting technical standards for the team. The position calls for substantial experience building reliable production software and shipping generative AI products that users depend on. The work takes place in a fast-moving, ambiguous AI product environment. The posting lists a base pay range of $176,100–$308,200 for positions in the stated location, plus equity when applicable, variable or incentive compensation, and benefits. Compensation varies with qualifications, skill level, competencies, and work location.

Requirements

At least 7 years of experience building production software systems, with a strong track record in reliability, performance, and scalability.
Hands-on experience shipping generative AI products and owning AI-powered features used by production users.
Solid understanding of how large language models work, including failure modes, context constraints, and how prompt design shapes model behavior at scale.
Experience with Python, Kubeflow, and SQL; experience with SageMaker, Vertex AI, or AzureML is also listed.
Practical prompt engineering experience, including systematically designing, versioning, and evaluating prompts across model updates or A/B evaluation cycles.
A track record in evaluation engineering, including evaluation suites designed, shipped, and used to guide production AI quality decisions.
Experience reasoning about model routing, inference cost, and latency tradeoffs in production.
Strong software engineering fundamentals, including distributed systems, API design, and testing discipline.
Comfort operating in fast-moving, ambiguous, startup-like AI product environments.

Nice To Haves

Experience with multi-agent coordination patterns such as A2A or MCP.
Familiarity with agent frameworks such as LangChain, LlamaIndex, or similar.
Prior experience shipping AI systems in enterprise software.
Experience with AI observability tooling, including tracing, cost tracking, and LLM-specific monitoring.
Familiarity with cloud-native infrastructure, service observability, logging, monitoring, reliability engineering, and production troubleshooting.

Responsibilities

Design and build the agent execution harness to route inputs, manage context, invoke tools, handle retries, and surface execution state across multi-step agent workflows.
Own runtime fault tolerance, latency, and throughput for enterprise workflows.
Instrument the harness with tracing, cost attribution, and latency visibility to support production diagnosis and identify failures.
Build prompt management systems for versioning, templating, and systematic evaluation across model updates and configuration changes.
Design and own evaluation frameworks, including unit evaluations, integration evaluations, and production monitors, to measure quality and catch regressions.
Integrate and abstract over frontier large language models, managing model routing, fallback strategies, cost, and latency tradeoffs.
Set technical standards through architecture decisions, code reviews, and coaching, particularly for agentic design patterns and production AI practices.
Define system boundaries for agent logic, including decisions about tool calls, sub-agents, hardcoded paths, and human escalation, and establish team-wide design standards.