Staff Research Engineer/Scientist

ServiceNow•Santa Clara, CA

About The Position

ServiceNow is seeking a Staff Research Engineer/Scientist to design and build data infrastructure and automated evaluation systems for agentic AI workflows. The role involves developing scalable data architectures and pipelines, creating tools and processes for reliable data transformation, and defining evaluation metrics and ground-truth standards for AI agent performance. The position will build automated evaluation pipelines using Now LLM Service models and execution logs, with dashboards, reporting, versioning, and reproducibility. It will establish quality standards and validation frameworks, improve system and workflow performance, and collaborate with product, engineering, and data science teams. The role also includes mentoring junior engineers and generalizing evaluation approaches across Agentic AI products.

Requirements

At least 9 years designing and building production data pipelines, data warehousing, and ETL systems.
Strong expertise in SQL, Python, JavaScript or equivalent, and distributed data processing.
At least 3 years working on ML systems, model validation infrastructure, automated testing, or AI-powered applications.
Hands-on experience with data quality, feature engineering, and model monitoring.
Proven experience deploying, monitoring, and maintaining large-scale data systems in production.
Experience troubleshooting data quality issues, latency, and system reliability.
Ability to design scalable, secure data architectures.
Experience with cloud platforms such as GCP or AWS, APIs, event-driven systems, and data governance patterns.
Familiarity with AI/ML evaluation frameworks, LLM-based systems, or agentic workflows, and ability to instrument and evaluate complex AI systems.
Understanding of model performance tracking.
Comfort driving technical conversations across teams, taking end-to-end ownership, working autonomously, and mentoring peers on data engineering standards and best practices.

Nice To Haves

Experience on the ServiceNow platform.

Responsibilities

Design data architecture and oversee deployment of pipelines to capture, manage, and store structured and unstructured data from internal and external sources.
Establish data processes and flows using cloud platforms, local databases, and applicable storage forms.
Develop technical tools using machine learning and data-engineering techniques to cleanse, organize, and transform data.
Automate processes that maintain data structure integrity and ongoing data quality standards.
Partner with product and AI teams to define evaluation metrics for agentic workflows, including task completeness, tool use, and workflow success.
Establish ground-truth labeling standards and validation criteria, and design evaluation datasets reflecting real-world agent execution.
Design and implement automated evaluation infrastructure to measure AI agent performance using Now LLM Service models and execution logs.
Create dashboards, reporting, versioning, and reproducibility for evaluation datasets and results.
Create design standards and quality-assurance processes for data systems, and define quality gates and validation frameworks.
Analyze workflow performance and recommend optimizations to address evolving CRM AI requirements.
Collaborate with product, engineering, and data science teams.
Mentor junior engineers on data engineering and evaluation design, and generalize evaluation patterns and metrics across Agentic AI products.