Senior Backend Engineer

LiteLLM•San Francisco, CA
•$200,000 - $260,000•Onsite

About The Position

LiteLLM is hiring senior and staff backend engineers to own the core of its open-source AI Gateway and keep it fast and reliable as traffic grows. LiteLLM is used by companies including Adobe, Netflix, NASA, and NVIDIA, and handles more than 1 billion requests per month. The role owns critical components of the LiteLLM proxy, including routing, authentication, caching, spend tracking, guardrails, and observability. The engineer will make these systems correct, fast, and dependable by diagnosing root causes, eliminating silent failures, and refactoring unreliable components without changing their behavior. They will also set a high standard for backend engineering practices on a mostly early-career team.

Requirements

  • At least 2 years building backend systems in production, with ownership of a mission-critical system such as payments, gateways, core APIs, or data stores.
  • Strong Python skills and experience with asynchronous or high-throughput services, such as asyncio, FastAPI, or similar.
  • A track record of improving production reliability through code, including refactors, re-architectures, migrations, or permanent root-cause fixes.
  • Excellent debugging skills across services, databases such as Postgres and Redis, and distributed systems.
  • High ownership and responsibility for the customer impact of shipped work.
  • BS/MS in Computer Science or a related field, or equivalent proof of exceptional ability.

Nice To Haves

  • Production experience with Rust, Go, or C++.
  • Experience at a fast-growing startup with a large, real codebase.
  • Experience with LLM APIs, proxies, or API gateways.
  • Open-source contributions.

Responsibilities

  • Own core proxy components end to end, from design through production, including handling incidents.
  • Find and eliminate classes of bugs, including race conditions, stale cache state across replicas, and errors silently swallowed instead of shown to users.
  • Ensure every guardrail and policy check, such as apply_guardrail, is logged and traceable and fails loudly when appropriate.
  • Build gateway-integrated guardrails that block common attacks on LLM APIs, MCP servers, and agents.
  • Keep observability integrations—including Datadog, Splunk, Prometheus, and OpenTelemetry—accurate at more than 1 billion requests per month, with minimal latency overhead and no memory leaks.
  • Help migrate performance-critical paths from Python to Rust without breaking behavior.
  • Raise the bar for the team on code review, testing, and debugging.

Benefits

  • $220,000–$250,000 base salary plus equity.
  • Health, dental, and vision benefits.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service