Staff Research Engineer/Scientist

ServiceNow•Santa Clara, CA

21h•$202,500 - $354,400

About The Position

Design and oversee data architecture and pipelines that capture, manage, and store structured and unstructured data from internal and external sources. Build tools and automated processes using machine learning and data-engineering techniques to cleanse, organize, transform, and maintain data quality. Partner with product and AI teams to define metrics, ground-truth labeling standards, validation criteria, and evaluation datasets for agentic workflows. Build automated evaluation infrastructure using Now LLM Service models and execution logs, with dashboards, reporting, versioning, and reproducibility for evaluation datasets and results. Establish design standards, quality assurance processes, quality gates, and validation frameworks for data systems. Analyze workflow performance and recommend improvements for evolving CRM AI requirements. Collaborate across product, engineering, and data science teams, mentor junior engineers, and generalize evaluation patterns and metrics across Agentic AI products.

Requirements

9+ years designing and building production data pipelines, data warehousing, and ETL systems.
Strong expertise in SQL, Python, JavaScript or equivalent, and distributed data processing.
3+ years working on ML systems, model validation infrastructure, automated testing, or AI-powered applications.
Hands-on experience with data quality, feature engineering, and model monitoring.
Proven track record deploying, monitoring, and maintaining large-scale data systems in production.
Experience troubleshooting data quality issues, latency, and system reliability.
Ability to design scalable, secure data architectures.
Experience with cloud platforms such as GCP or AWS, APIs, event-driven systems, and data governance patterns.
Familiarity with AI/ML evaluation frameworks, LLM-based systems, or agentic workflows.
Ability to instrument and evaluate complex AI systems, with an understanding of model performance tracking.
Comfortable driving technical conversations across multiple teams, taking end-to-end ownership, working autonomously, and mentoring peers on data engineering standards and best practices.

Nice To Haves

Experience on the ServiceNow platform.

Responsibilities

Design and oversee deployment of data architecture and pipelines for structured and unstructured data from internal and external sources.
Establish data processes and flows using cloud platforms, local databases, and applicable storage forms.
Develop tools using machine-learning and data-engineering techniques to cleanse, organize, and transform data.
Automate ongoing processes that maintain data structure integrity and quality standards.
Partner with product and AI teams to define evaluation metrics for agentic workflows, including task completeness, tool use, and workflow success.
Establish ground-truth labeling standards and validation criteria, and design evaluation datasets that reflect real-world agent execution.
Design and implement automated evaluation infrastructure to measure AI agent performance using Now LLM Service models and execution logs.
Create dashboards, reporting, versioning, and reproducibility for evaluation datasets and results.
Create design standards and quality assurance processes for data systems, including quality gates and validation frameworks.
Analyze workflow performance and recommend optimizations for evolving CRM AI requirements.
Collaborate with product, engineering, and data science teams.
Mentor junior engineers on data engineering and evaluation design.
Generalize evaluation patterns and metrics across Agentic AI products.