Staff AI Engineer: Foundational AI Services

ServiceNow • Addison, TX

8h • Remote

About The Position

We are looking for a visionary Staff AI Engineer to architect and scale our core AI intelligence layer. In this role, you won't just be building features; you will be building the "AI Operating System" for our company—the foundational services that empower every product team to deploy production-grade GenAI. The Mission As a Staff Engineer, you will own the end-to-end lifecycle of our internal AI infrastructure, moving beyond simple wrappers to create high-performance, resilient, and autonomous systems.

Requirements

• Typically requires 8+ years of overall software engineering experience.
• Core AI: Expert-level mastery of Transformers, attention mechanisms, and the latest frontier models (GPT-4o, Claude 3.5, Llama 3).
• The Stack: Deep experience with vector databases (Pinecone, Weaviate, Milvus), orchestration layers (LangChain, LlamaIndex), and MLOps tools.
• Software Craft: You are a Staff-level coder in Python/Rust who understands distributed systems, concurrency, and API design.
• Modern Buzz: You live and breathe Chain-of-Thought (CoT), DSPy, GraphRAG, and Semantic Caching.

Responsibilities

• LLM Fine-Tuning & Distillation: Lead domain-specific model optimization using PEFT (LoRA/QLoRA) and knowledge distillation to balance cost, latency, and reasoning capability.
• Architect Foundational RAG: Build next-gen Retrieval-Augmented Generation pipelines using hybrid search, cross-encoders, and self-correcting retrieval loops.
• Agentic Orchestration: Design and deploy multi-agent systems using frameworks like LangGraph or CrewAI, enabling autonomous task planning and tool-use (Function Calling).
• Enterprise-Grade Evaluation: Build "LLM-as-a-judge" frameworks and robust eval pipelines to measure hallucination rates, groundedness, and safety.
• Inference Optimization: Implement high-throughput, low-latency serving strategies including quantization, speculative decoding, and prompt caching.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed
• Number of Employees: 5,001-10,000 employees