Staff AI Engineer, Agentic AI Harness & Quality - Moveworks
About The Position
The Staff AI Engineer will join the Agentic AI Harness & Quality team to expand ServiceNow’s customer-facing agentic AI capabilities. The role applies software engineering and compound AI system engineering to develop and maintain high-performing AI systems, with a focus on agent quality, reliability, latency, and end-to-end user experience. The team works with frontier LLMs, tools, and infrastructure for tuning, evaluating, and serving agents in production, supported by an annotation team that helps create high-quality, inclusive, and privacy-preserving datasets. The engineer will help improve agent performance and customer value through data-centric, evaluation-driven work. This includes researching and applying AI advances, experimenting with changes to models and system components, and scaling successful solutions into production. The role collaborates closely with web and chat platform engineering teams and other functions across Moveworks and ServiceNow to deliver effective customer experiences.
Requirements
- Experience as a software engineer building AI-native products or platforms.
- Ability to ship production-quality, fully unit-tested code and rigorously evaluated system or model updates.
- Solid understanding of LLM and agent evaluation fundamentals, especially evaluation of agent action trajectories, generated text, and non-uniform sampling regimes.
- Strong attention to detail and high standards for evaluation dataset quality.
- Thoughtfulness about API contracts and the implications of exposing platform features to stakeholders or end users.
- Ability to work in a Mac development environment and program in Python and/or Golang, with coding-agent assistance.
- Familiarity with industry-standard agent architectures and large language models.
- Strong English-language reading skills to identify subtle miscommunication bugs in agent instructions, and ability to address them through precise writing and evaluations.
- Ownership and drive to move quickly and deliver product improvements.
- Curiosity about engineering beyond the immediate discipline and an ongoing desire to learn and stay current in agentic AI.
Responsibilities
- Apply software engineering and compound AI system engineering to build lasting value for customers.
- Improve agent behavior by increasing success at user goals and reducing misbehavior in customer business environments.
- Address challenges in AI harness engineering, including agent cognitive architecture, evaluation benchmark creation, context engineering, generative UI, dynamic orchestration, multimodal input and output, multilinguality, conversational memory, reasoning, grounding and verifiability, deployment safety, and automatic harness optimization.
- Read, discuss, and build on current AI research, open-source repositories, and models.
- Research and develop innovative, scalable, and dynamic solutions to difficult problems.
- Use machine learning and LLM knowledge to improve the agentic harness and its AI components, evaluate solutions through small-scale experiments, and productionize them at scale.
- Ship product improvements using production-quality, fully unit-tested code and rigorously evaluated updates to models, prompts, context curation code, and other tunable system components.
- Partner with web and chat platform engineering teams to turn harness investments into customer-facing user experiences.
Benefits
- Health plans
- Flexible spending accounts
- 401(k) plan with company match
- Employee stock purchase plan
- Matching donations
- Flexible time away plan
- Family leave programs
- Equity, when applicable
- Variable/incentive compensation
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
