LiteLLM is hiring a Staff Backend Engineer to own critical parts of its proxy, including routing, authentication, caching, spend tracking, guardrails, and observability. The role focuses on making these systems correct, fast, and dependable at more than 1 billion requests per month. This includes finding root causes of silent failures, fixing them in code, and refactoring fragile components without changing their behavior. The engineer will also help set standards for backend code, testing, and delivery on a mostly early-career team. The role includes owning core proxy components from design through production and incident response, building guardrails against common attacks on LLM APIs, MCP servers, and agents, and maintaining accurate observability integrations with minimal latency overhead and no memory leaks. It also involves migrating performance-critical paths from Python to Rust while preserving behavior. The position is onsite in San Francisco. LiteLLM describes itself as a Python SDK and proxy server for calling more than 100 LLM APIs in the OpenAI format.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Job Type
Full-time
Career Level
Senior