Machine Learning Engineer, Agentic AI Harness & Quality - Moveworks

ServiceNow•Mountain View, CA

5d•Remote

About The Position

Moveworks is seeking an experienced software engineer with machine learning expertise to join the Agentic AI Harness & Quality team. This role will focus on expanding Moveworks' agentic AI capabilities, enhancing user experiences, and improving the platform's AI performance. The team utilizes modern agentic AI tools, including LLMs, multimodal foundation models, and hybrid vector databases, along with infrastructure for tuning, evaluation, and production serving. As an eval-driven, data-centric team, you will collaborate with a world-class annotation team to build high-quality, inclusive, and privacy-preserving datasets. The role goes beyond model training to achieve state-of-the-art AI performance in production, encompassing accuracy, quality, latency, reliability, and overall system capability. Successful engineers are motivated to design and maintain high-performing compound AI systems, with a focus on speed, solving challenging product and engineering problems, and advancing agent reliability and customer value. This work directly contributes to building the highest-performing enterprise assistant platform, in collaboration with Moveworks and ServiceNow teams.

Requirements

Drive to ship product improvements with production-quality, fully unit-tested code and rigorously-evaluated updates to models, prompts, or other tunable system components.
Ability to solve problems end-to-end with machine learning.
Solid grasp of model evaluation fundamentals, especially for text generation, text classification, and non-uniform sampling regimes.
Attention to detail and high standard of data quality for training and especially evaluation datasets.
Readiness to hit the ground running in a Mac development environment, programming in Python and/or Golang.
Knowledge of deep learning architectures and algorithms and leading large language models.
Desire to work at a startup pace in a medium-sized company with a high degree of ownership.
Drive to ship product improvements with production-grade code.
Strong appetite for continuous incremental wins and completing challenging projects fast.
High level of curiosity about engineering outside of immediate discipline and ongoing desire to learn and stay at the cutting edge of NLU & AI.

Responsibilities

Apply software engineering, machine learning, and compound AI system engineering to create lasting value for customers.
Drive quality in agent behavior, increasing utility and decreasing misbehavior.
Tackle challenges in AI harness engineering, including agent cognitive architecture, eval benchmark creation, context engineering, generative UI, dynamic agent orchestration, multimodal I/O, multilingual support, conversational memory management, reasoning strategies, abstractive summarization, grounding and verifiability for generated text, deployment safety, and self-learning.
Read, discuss, and build on the latest ML/LLM research and open-source repositories and models.
Research and develop innovative, scalable, and dynamic solutions to complex problems.
Use machine learning fundamentals and LLMs to improve the agentic harness and its AI components, evaluate them with experiments, and productionize solutions at scale.
Partner closely with web and chat platform engineering teams to translate harness investments into delightful user experiences.