Staff Backend Engineer

LiteLLM•San Francisco, CA
•$250,000 - $350,000•Onsite

About The Position

LiteLLM is hiring a Staff Backend Engineer to own critical parts of its proxy, including routing, authentication, caching, spend tracking, guardrails, and observability. The role focuses on making these systems correct, fast, and dependable at more than 1 billion requests per month. This includes finding root causes of silent failures, fixing them in code, and refactoring fragile components without changing their behavior. The engineer will also help set standards for backend code, testing, and delivery on a mostly early-career team. The role includes owning core proxy components from design through production and incident response, building guardrails against common attacks on LLM APIs, MCP servers, and agents, and maintaining accurate observability integrations with minimal latency overhead and no memory leaks. It also involves migrating performance-critical paths from Python to Rust while preserving behavior. The position is onsite in San Francisco. LiteLLM describes itself as a Python SDK and proxy server for calling more than 100 LLM APIs in the OpenAI format.

Requirements

  • At least 5 years of experience building backend systems in production.
  • Experience owning a mission-critical system, such as payments, gateways, core APIs, or data stores.
  • Strong Python skills and experience with async or high-throughput services, such as asyncio, FastAPI, or similar.
  • A track record of improving production-system reliability through refactors, re-architectures, migrations, or lasting root-cause fixes.
  • Excellent debugging skills across services, databases such as Postgres and Redis, and distributed systems.
  • High ownership and responsibility for the customer impact of shipped work.
  • BS/MS in Computer Science or a related field, or equivalent proof of exceptional ability.

Nice To Haves

  • Production experience with Rust, Go, or C++.
  • Experience at a fast-growing startup with a large, real codebase.
  • Experience with LLM APIs, proxies, or API gateways.
  • Open-source contributions.

Responsibilities

  • Own core proxy components end to end, from design through production, including incident response.
  • Find and eliminate classes of bugs, including race conditions, stale cache state across replicas, and errors silently swallowed instead of surfaced to users.
  • Ensure guardrails and policy checks, such as apply_guardrail, are logged, traceable, and fail loudly when appropriate.
  • Build gateway-integrated guardrails that block common attacks on LLM APIs, MCP servers, and agents.
  • Maintain accurate Datadog, Splunk, Prometheus, and OpenTelemetry integrations at more than 1 billion requests per month, with minimal latency overhead and no memory leaks.
  • Help migrate performance-critical paths from Python to Rust without changing behavior.
  • Raise the team's standards for code review, testing, and debugging.

Benefits

  • $250,000–$350,000 base salary plus equity.
  • Health, dental, and vision coverage.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service