Senior Staff Machine Learning Engineer - DevOps

ServiceNow•Santa Clara, CA

18h•Hybrid

About The Position

ServiceNow is seeking a Senior Staff Machine Learning Engineer (DevOps) for its AI Services Platform Group. The team builds and evolves the AI platform, partners with other teams on AI-powered products and work experiences, and researches and de-risks technologies for future experiences. This role leads the architecture, optimization, and operational excellence of the AI inference platform. The engineer owns the end-to-end inference lifecycle, from onboarding models to deploying optimized inference engines in production across multiple backends on Kubernetes. The role also architects an intelligent gateway layer using Envoy and ext_proc for access control, observability, and advanced request routing. It combines expertise in GPU compute, LLM APIs, inference frameworks, containers, orchestration, and API gateway architecture with a focus on practical engineering and measurable performance outcomes. The role requires working from the Santa Clara office two days per week.

Requirements

Expertise in Go or Python, object-oriented programming, design patterns, and time- and space-efficient algorithms.
Experience building new products that use challenging algorithms.
Expertise in writing efficient, object-oriented, modularized, high-quality software.
Knowledge of core AI/ML techniques and algorithms.
Knowledge of unit testing, profiling, and code tuning.
At least 8 years of backend software engineering experience.
At least 4 years of experience designing and operating distributed systems.
At least 3 years of Kubernetes experience in production, including Helm and CI/CD.
Bachelor’s degree in computer science, software engineering, or a closely related technical field; equivalent practical experience may be considered where technical depth is clearly demonstrated.
Proficiency using AI coding assistants such as GitHub Copilot or Cursor, with the ability to identify logic errors, security anti-patterns, and edge cases in generated code.
Experience with AI-assisted debugging and trace analysis, and understanding of data privacy rules governing prompt contents.

Responsibilities

Lead the architecture, optimization, and operational excellence of the AI inference platform.
Own the inference lifecycle, from onboarding models to deploying optimized inference engines in production across multiple backends on Kubernetes.
Architect an intelligent gateway layer using Envoy and ext_proc to enforce access control, provide observability, and enable advanced request routing.
Deploy, scale, and operate services in production Kubernetes environments, including Helm-based deployments and CI/CD pipelines that support repeatable releases and safe rollbacks.
Build and own observability through meaningful metrics, useful logs, tracing, and alerts focused on customer impact.
Independently debug and resolve production incidents, participate in on-call, conduct root-cause analysis, and operate against defined service level objectives.
Use AI coding assistants and AI-assisted debugging and trace analysis while reviewing generated code rigorously and following data privacy rules.