Senior Staff Software Engineer

ServiceNow•San Diego, CA

4h•$171,900 - $300,800

About The Position

ServiceNow is seeking a Senior Staff Software Engineer to lead the team responsible for the Build Agent evaluation framework, model support, and telemetry. Build Agent is ServiceNow’s AI coding assistant, designed to operate across the platform’s scoped applications, tables, and metadata types. The evaluation framework has helped improve correctness, reduce latency and inference costs, and identify serious defects before production. The team has eight engineers, and the role will help scale this initiative into a durable function. The role owns the evaluation strategy and roadmap, including prompt-set design, scoring methodology, failure-mode taxonomy, and telemetry instrumentation. It coordinates quality standards and follow-through across the teams contributing to Build Agent, leads structured model benchmarking and support recommendations, and addresses ambiguous problems involving evaluation signals, model capability, and platform architecture. The position offers a base pay range of $171,900–$300,800, plus equity when applicable, variable or incentive compensation, and benefits. Compensation varies with qualifications, skills, competencies, and work location.

Requirements

Direct experience building or leading evaluation systems for LLM-based products, including scoring methodology, benchmark design, and failure analysis.
Understanding of coding agent architecture, including tool use, agentic loops, context management, and trade-offs between generic and domain-specific scaffolding.
At least 6 years of experience with technologies relevant to ServiceNow.
Advanced coding skills and fluency in one or more of Java, C++, Ruby, Shell, or JavaScript.
Experience critically evaluating foundation models and distinguishing capability differences from tuning or scaffold artifacts.
Ability to execute against ambiguous priorities by weighing context, risk, and desired outcomes, particularly when evaluation data is incomplete or contested.

Responsibilities

Own the evaluation strategy and roadmap, including prompt-set design, scoring methodology, failure-mode taxonomy, and telemetry instrumentation.
Coordinate across teams contributing to Build Agent to establish quality standards and drive evaluation findings to committed, owned fixes.
Lead model benchmarking and make data-backed recommendations on model support decisions.
Address ambiguous, cross-cutting problems involving evaluation signals, model capability, and platform architecture.
Develop evaluation infrastructure covering golden prompt sets, Pass@1 and functional scoring, failure categorization, ServiceNow metadata types, and UI workflows.
Support structured evaluation of candidate foundation models against the production default, including failure-consistency analysis.
Track token usage, cache efficiency, and inference cost alongside correctness as release signals.
Prioritize evaluation coverage across surfaces owned by different teams and convert evaluation signals into fixes.