Backend LLM Engineer

Berrie AI•San Jose, CA

About The Position

LiteLLM is hiring a Backend LLM Engineer to help maintain and extend its open-source LLM Gateway, which provides a Python SDK and FastAPI server for calling more than 100 LLM APIs in the OpenAI format. The engineer will focus on unifying APIs across major LLM providers, including OpenAI, Gemini, and Anthropic, and will work directly with the CEO and CTO on critical projects. The role involves translating requests between OpenAI and Anthropic specifications and the formats used by individual LLM providers, as well as developing provider-agnostic functionality. Example projects include session management across non-OpenAI models for the /v1/responses API, support for Anthropic and Bedrock's Anthropic “thinking” parameter, handling provider-specific limitations such as OpenAI o-1 streaming, maintaining unified APIs across multiple endpoints and providers, and implementing Anthropic cost tracking and logging. The tech stack includes Python, FastAPI, Redis, and Postgres.

Requirements

  • 1–2 years of backend or full-stack experience with production systems.
  • Experience working with the OpenAI API, including understanding the difference between /chat/completions and /responses and being able to discuss API-specific nuances.
  • Passion for open source and user engagement.
  • Strong work ethic and ability to thrive in a small team.
  • Eagerness to talk to users and help solve real problems.
  • Skills in Python, LLM APIs, FastAPI, and high-throughput/low-latency systems.

Responsibilities

  • Ensure LiteLLM unifies the format for calling LLM APIs across the broader OpenAI and Anthropic specifications.
  • Build transformations that convert OpenAI- or Anthropic-format requests into provider-specific formats.
  • Develop provider-agnostic functionality, such as session management across non-OpenAI models for the /v1/responses API.
  • Add support for Anthropic and Bedrock Anthropic's “thinking” parameter.
  • Handle provider-specific quirks, including OpenAI o-1 streaming limitations.
  • Maintain unified APIs across /v1/messages, /v1/responses, and /chat/completions for OpenAI, Gemini, and Anthropic models across Azure, OpenAI API, Bedrock Invoke, Bedrock Converse, Vertex AI, and Google AI Studio.
  • Implement cost tracking and logging for the Anthropic API.
  • Engage with users and help solve their problems.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service