Staff AI Engineer, Agentic AI Harness & Quality - Moveworks

ServiceNow • Mountain View, CA

1d • Onsite

About The Position

Moveworks, now part of ServiceNow, is seeking an experienced software engineer with expertise in building AI-native products and platforms to expand customer-facing agentic AI capabilities. As a member of the Agentic AI Harness & Quality team, the engineer will work with frontier LLMs, agent tools, and infrastructure for tuning, evaluating, and serving production AI agents. The team uses an evaluation-driven, data-centric approach and works with an annotation team to create high-quality, inclusive, and privacy-preserving datasets.

The role focuses on improving production performance across output quality and accuracy, latency, reliability, and end-to-end user-facing capabilities. The engineer will design and maintain high-performing compound AI systems, contributing to the goal of building a leading enterprise assistant platform in collaboration with teams across Moveworks and ServiceNow.

Requirements

• Demonstrated drive to ship product improvements using production-quality, fully unit-tested code and rigorously evaluated updates to models, prompts, context curation code, or other tunable system components.
• Solid understanding of LLM and agent evaluation fundamentals, particularly for agent action trajectories, generated text, and non-uniform sampling regimes.
• Attention to detail and high standards for evaluation dataset quality.
• Thoughtfulness about API contracts and the implications of exposing platform features to different stakeholders or end users.
• Ability to work in a Mac development environment and program in Python and/or Golang, with assistance from coding agents.
• Familiarity with industry-standard agent architectures and large language models.
• Critical reading skills to identify subtle miscommunication bugs in English-language agent instructions.
• Ability to fix miscommunication bugs through precise writing and evaluation.
• Desire to move quickly with a high degree of ownership.
• Drive to ship product improvements with production-grade code.
• Strong appetite for continuous incremental wins and completing challenging projects quickly.
• Curiosity about engineering disciplines beyond the immediate area of work and an ongoing desire to learn and stay current with agentic AI.

Responsibilities

• Apply software engineering and compound AI system engineering to create lasting value for customers.
• Improve agent behavior by increasing success rates in achieving end-user goals and reducing misbehavior in customer business environments.
• Address AI harness engineering challenges, including agent cognitive architecture, evaluation benchmark creation, context engineering, generative UI, dynamic agent orchestration, multimodal image and voice input/output, multilinguality, conversational memory management, reasoning strategies, grounding and verifiability, deployment safety, and automatic harness optimization.
• Read, discuss, and build on current AI research, open-source repositories, and models.
• Research and develop innovative, scalable, and dynamic solutions to difficult problems.
• Use machine learning and LLM knowledge to improve the agentic harness and its AI components, evaluate changes through small-scale experiments, and productionize solutions at scale.
• Partner with web and chat platform engineering teams to turn harness investments into customer-facing user experiences.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed