Staff Engineer, AI

LiteLLM•San Francisco, CA

19h•$250,000 - $350,000•Onsite

About The Position

LiteLLM is hiring a hands-on technical lead to lead its unified API team and ensure that new models, endpoints, and providers work with LiteLLM on day one. The role owns how LiteLLM unifies LLM APIs across OpenAI and Anthropic specifications, leads the engineers building that functionality, and sets technical and code-quality standards. The position also owns developer experience for LiteLLM’s Python SDK and proxy and includes working directly with customers and power users to determine product priorities. The role reports to the CEO initially and later to the Head of Engineering. It is in person in San Francisco; relocation assistance and H-1B transfer support are available. The technology stack includes Python, FastAPI, Redis, and Postgres, with the gateway moving to Rust. LiteLLM provides a unified API for more than 100 LLM providers, as well as routing, authentication, budgets, observability, and governance for production AI systems.

Requirements

At least 3 years of experience; the scope owned matters more than having held a Staff title.
Experience as a machine learning engineer who now builds applied AI products.
Deep expertise in LLM APIs, including differences between /chat/completions and /responses and individual provider nuances.
Experience building or maintaining a unified API in front of multiple providers.
Strong software engineering fundamentals in Python.
Good judgment about developer experience and ability to explain what makes an API or playground poor.
Customer empathy, including asking why users want something and assessing the size of the use case before building it.
Readiness to lead a team from day one and set technical standards.

Nice To Haves

Experience at a company built on a unified API, such as OpenRouter, Vercel AI SDK, Merge, Segment, or Fivetran.
Having built a unified API or written about what a good one should do.
Experience building evaluations for LLMs.
Experience using LiteLLM in production.
Open-source contributions, especially to LLM tooling.

Responsibilities

Lead the unified API team and own new model, endpoint, and provider rollouts end to end.
Design provider-agnostic gateway features, such as session management for non-OpenAI models.
Triage open-source issues and external pull requests involving new providers and translation fixes, and get them merged, reviewed, or replaced with an internal fix.
Review the AI team's pull requests and maintain a high bar for code quality, including code written with AI.
Identify power users in the role's area and use their needs to guide what to build next.
Take on new unification bets with a customer who will use them and provide feedback.
Own the developer experience for users of LiteLLM's Python SDK and proxy.
Lead the team from day one and set the technical bar.