Staff Machine Learning Engineer, Agentic AI Harness & Quality - Moveworks

ServiceNow • Mountain View, CA

5d

About The Position

Moveworks is seeking an experienced software engineer with machine learning expertise to join the Agentic AI Harness & Quality team. This role focuses on expanding Moveworks' agentic AI capabilities, enhancing user experiences, and improving the platform's AI performance. The team utilizes modern agentic AI engineering tools, including LLMs, multimodal foundation models, and hybrid vector databases, along with infrastructure for tuning, evaluation, and production serving. The team is data-centric and eval-driven, supported by an annotation team for building error-free, inclusive, and privacy-preserving datasets. The role goes beyond model training to achieve state-of-the-art AI performance in production, encompassing accuracy, quality, latency, reliability, and the overall end-to-end user-facing system. Successful engineers design and maintain high-performing compound AI systems, with model training as one of several tools. The team's core objective is to build the highest-performing enterprise assistant platform, collaborating closely with other functions within Moveworks and ServiceNow.

Requirements

• Drive to ship product improvements with production-quality, fully unit-tested code and rigorously-evaluated updates to models, prompts, or other tunable system components.
• Ability to solve problems end-to-end with machine learning.
• Solid grasp of model evaluation fundamentals, especially for agent trajectories, text generation, text classification, and non-uniform sampling regimes.
• Attention to detail and high standard of data quality for training and especially evaluation datasets.
• Readiness to hit the ground running in a Mac development environment, programming in Python and/or Golang.
• Familiarity with deep learning architectures and algorithms and leading large language models.
• Desire to work at a startup pace in a medium-sized company with a high degree of ownership.
• Drive to ship product improvements with production-grade code.
• Strong appetite for continuous incremental wins and completing challenging projects fast.
• High level of curiosity about engineering outside of immediate discipline and ongoing desire to learn and stay at the cutting edge of NLU & AI.

Responsibilities

• Apply software engineering, machine learning, and compound AI system engineering to create lasting value for customers.
• Drive quality in agent behavior, increasing utility for end-users and decreasing misbehavior.
• Tackle challenges in AI harness engineering, including agent cognitive architecture, eval benchmark creation, context engineering, generative UI, dynamic agent orchestration, multimodal I/O, multilinguality, conversational memory management, reasoning strategies, abstractive summarization, grounding and verifiability for generated text, deployment safety, and self-learning.
• Read, discuss, and build on the latest ML/LLM research and open-source repositories and models.
• Research and develop innovative, scalable, and dynamic solutions to complex problems.
• Use machine learning fundamentals and LLMs to improve the agentic harness and its AI components, evaluate them with experiments, and productionize solutions at scale.
• Partner closely with web and chat platform engineering teams to translate harness investments into user experiences.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed