Sr. Software Engineer, Core AI

Navan • San Francisco, CA

1d

About The Position

As a Senior Software Engineer, Core AI, you will design, build, evaluate, and operate production-grade AI services and agents that improve high-value human workflows across the company. You will own work end to end, from data and context design through tool use, application logic, evaluation, and production monitoring.

You will build specialized, task-specific AI services and agents; define and evaluate expected behavior; use evaluation findings and production feedback to improve quality; and incorporate tracing, observability, versioning, monitoring, reliability practices, and success metrics. You will run controlled experiments to compare models, data, and implementation approaches, partner with AI Architecture, AI Ops, and Platform teams on secure and scalable service architecture, and ship iteratively from user problem to production.

Requirements

• At least 5 years of strong software engineering experience delivering production services or products.
• Hands-on experience building AI/LLM-powered applications or agents used by real users or in operational settings.
• Experience with evaluation methods for AI behavior, including defining expected outcomes, building test sets or evaluations, tracking regressions, and acting on results.
• Ability to shape retrieval, context, and serialization so models receive the information needed for a task.
• Good judgment about tool use, autonomy, human review, failure handling, privacy, reliability, latency, and cost.
• A product-minded approach focused on solving specific workflow problems and measuring whether solutions work.
• Comfort learning tools and frameworks.

Nice To Haves

• Experience at a senior or staff level.
• Experience with agent runtimes, model APIs, MCP, tracing, or evaluation tools.

Responsibilities

• Design and implement specialized, task-specific AI services and agents, including data serialization, retrieval, memory where appropriate, tool integrations, and bounded multi-step reasoning.
• Work with partners to define expected behavior and create and maintain evaluations using representative examples.
• Use evaluation results to measure performance against expectations and track regressions.
• Analyze evaluation findings, failures, and production feedback to improve context and data, service logic, tools, and model behavior.
• Build production services with tracing, observability, versioning, monitoring, reliability practices, and success metrics.
• Run repeatable, controlled benchmarks to compare models, data, and implementation approaches and make improvements attributable and reproducible.
• Partner with AI Architecture, AI Ops, and Platform teams to make services secure, reliable, cost-aware, and deployable at scale.
• Move from user problem to prototype to production iteratively, learning from real workflow outcomes.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed