LiteLLM is hiring a Backend LLM Engineer to help maintain and extend its open-source LLM Gateway, which provides a Python SDK and FastAPI server for calling more than 100 LLM APIs in the OpenAI format. The engineer will focus on unifying APIs across major LLM providers, including OpenAI, Gemini, and Anthropic, and will work directly with the CEO and CTO on critical projects. The role involves translating requests between OpenAI and Anthropic specifications and the formats used by individual LLM providers, as well as developing provider-agnostic functionality. Example projects include session management across non-OpenAI models for the /v1/responses API, support for Anthropic and Bedrock's Anthropic “thinking” parameter, handling provider-specific limitations such as OpenAI o-1 streaming, maintaining unified APIs across multiple endpoints and providers, and implementing Anthropic cost tracking and logging. The tech stack includes Python, FastAPI, Redis, and Postgres.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Job Type
Full-time
Career Level
Entry Level
Education Level
No Education Listed