Staff Engineer, AI

LiteLLM • San Francisco, CA

16h • Onsite

About The Position

LiteLLM is hiring a hands-on technical lead to lead its unified API team. The role owns how LiteLLM unifies LLM APIs across OpenAI and Anthropic specifications, leads the engineers building that functionality, and sets the technical quality bar. The goal is to make new models, endpoints, modalities, features, and providers available through LiteLLM on day one, while reducing open-source translation issues and ensuring developers can rely on the platform to keep pace with provider changes.

The position leads the open-source pod, where many issues involve translating between LLM APIs, and owns developer experience for LiteLLM’s Python SDK and proxy. The role reports to the CEO initially and later to the Head of Engineering. It is an in-person position in San Francisco; relocation help and H-1B transfer support are available.

The technology stack includes Python, FastAPI, Redis, and Postgres, and the gateway is moving to Rust.

Requirements

• At least 3 years of experience.
• Experience as a machine learning engineer and now building applied AI products.
• Deep expertise in LLM APIs, including the differences between /chat/completions and /responses and the nuances of individual providers.
• Experience building or maintaining a unified API that sits in front of many providers.
• Strong software engineering fundamentals in Python.
• Good judgment about developer experience and the ability to explain when an API or playground is poor.
• Customer empathy, including asking why users want something and assessing the scale of the use case before building it.
• Ability to lead a team from day one and set the technical bar.

Nice To Haves

• Experience at a company built on a unified API, such as OpenRouter, Vercel AI SDK, Merge, Segment, or Fivetran.
• Experience building a unified API or writing about what a good unified API should do.
• Experience building evaluations for LLMs.
• Experience using LiteLLM in production.
• Open-source contributions, especially to LLM tooling.

Responsibilities

• Lead the unified API team and own new model, endpoint, and provider rollouts end to end.
• Design provider-agnostic gateway features, such as session management for non-OpenAI models.
• Triage open-source issues and external pull requests involving new providers and translation fixes, and get them merged, reviewed, or replaced with an internal fix.
• Review the AI team’s pull requests and maintain a high code-quality bar, including for code written with AI.
• Identify power users in the area and use their needs to decide what to build next.
• Take on new unification bets with a customer who will use them and provide feedback.
• Own the developer experience for users of LiteLLM’s Python SDK and proxy.

Benefits

• Relocation assistance.
• Support for H-1B transfers.
• Competitive salary and equity.
• Health, dental, and vision benefits.
• Work directly with the founders every day.
• Own a whole area of the product; engineers talk to customers and decide what to build without product managers.
• Work ships in open source and is used by engineering teams around the world.
• Move quickly and see work in production immediately.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed