Staff AI Engineer, Agentic AI Harness & Quality - Moveworks

ServiceNow • Mountain View, CA

1d

About The Position

Moveworks is hiring an experienced software engineer to expand ServiceNow’s customer-facing agentic AI capabilities as part of the Agentic AI Harness & Quality team. The role focuses on building and maintaining high-performing compound AI systems using frontier large language models, tools, and infrastructure for tuning, evaluating, and serving agents in production. The team is evaluation-driven and data-centric, with support from an annotation team to develop high-quality, inclusive, and privacy-preserving datasets.

The engineer will improve production performance across output accuracy and quality, latency, reliability, and end-to-end system capability. The work includes advancing agent reliability and customer value, developing scalable solutions to complex AI and engineering problems, and collaborating across Moveworks and ServiceNow to build an enterprise assistant platform. The role also partners with web and chat platform engineering teams to turn harness improvements into strong customer experiences.

Requirements

• Experience as a software engineer building AI-native products or platforms.
• Ability to ship product improvements using production-quality, fully unit-tested code and rigorously evaluated system updates.
• Solid understanding of LLM and agent evaluation fundamentals, especially evaluation of agent action trajectories, generated text, and non-uniform sampling regimes.
• Strong attention to detail and high standards for evaluation dataset quality.
• Thoughtfulness about API contracts and the implications of exposing platform features to different stakeholders or end users.
• Ability to work in a Mac development environment and program in Python and/or Golang, using coding agents.
• Familiarity with industry-standard agent architectures and large language models.
• Critical reading skills to identify subtle miscommunication bugs in English-language agent instructions.
• Ability to resolve instruction miscommunication bugs through precise writing and evaluation.
• Ownership, speed, and motivation to deliver product improvements and complete challenging projects quickly.
• Curiosity about engineering beyond the immediate discipline and commitment to ongoing learning in agentic AI.

Responsibilities

• Apply software engineering and compound AI system engineering to create lasting value for customers.
• Improve agent behavior by increasing success at user goals and reducing misbehavior in customer business environments.
• Address AI harness challenges including agent cognitive architecture, evaluation benchmark creation, context engineering, generative UI, dynamic agent orchestration, multimodal input and output, multilinguality, conversational memory, reasoning strategies, grounding and verifiability, deployment safety, and automatic harness optimization.
• Read, discuss, and build on current AI research, open-source repositories, and models.
• Research and develop innovative, scalable, and dynamic solutions to difficult problems.
• Apply machine learning and LLM knowledge to improve the agentic harness and its AI components, test solutions through small-scale experiments, and productionize them at scale.
• Work closely with web and chat platform engineering teams to deliver customer-facing experiences based on harness improvements.
• Ship production-quality, unit-tested code and rigorously evaluated updates to models, prompts, context-curation code, and other tunable system components.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed