ML Engineer

LiteLLM•San Francisco, CA
•$200,000 - $250,000

About The Position

LiteLLM is hiring an ML Engineer to help build its next generation of AI infrastructure and its second product. The role focuses on unifying how applications call LLM APIs across the broader OpenAI and Anthropic specifications. This includes transforming OpenAI- and Anthropic-format requests into provider-specific formats and developing provider-agnostic capabilities, such as session management for non-OpenAI models through the /v1/responses API. The ML Engineer will work directly with the CEO and CTO on system design for bringing provider-level features to the gateway, including skills, compaction, and batches. The role also involves improving the developer experience for users of LiteLLM's Python SDK and maintaining interoperability across LLMs, MCPs, and agents, including supporting MCP authentication flows and agent use cases. LiteLLM's tech stack includes Python, FastAPI, Redis, and Postgres.

Requirements

  • 1–2 years of backend or full-stack experience with production systems.
  • Experience working with the OpenAI API, including understanding the differences between /chat/completions and /responses and API-specific nuances.
  • Passion for open source and user engagement.
  • Strong work ethic and ability to thrive in small teams.
  • Eagerness to talk to users and help solve real problems.
  • Skills: Python, LLM APIs, and FastAPI.

Responsibilities

  • Develop transformations that convert OpenAI- and Anthropic-format API requests into formats supported by different LLM providers.
  • Build provider-agnostic functionality, including session management across non-OpenAI models for the /v1/responses API.
  • Design systems for supporting provider-level features at the gateway, such as skills, compaction, and batches.
  • Improve the developer experience for users of LiteLLM's Python SDK.
  • Maintain LLM interoperability across LLMs, MCPs, and agents.
  • Support a wide range of MCP authentication flows and enable agent use cases.
  • Engage with users and help solve real problems.

Benefits

  • Competitive salary
  • Equity
  • Health benefits
  • Dental benefits
  • Vision benefits
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service