Litelm: LiteLLM Without the Bloat — A Leaner LLM Routing Library

TL;DR: A Leaner Path to Multi-Provider LLM Calls

Litelm is a new open-source Python library that tackles a common developer pain point: LiteLLM's feature creep. While LiteLLM has become a popular gateway for routing requests to multiple LLM providers, its codebase has grown to over 100,000 lines of code, bundling a proxy server, caching, cost tracking, and a host of other features that many developers never use. Litelm strips that down to the essentials—model routing, message translation, streaming, tool use, and embeddings—in about 2,900 lines of code with just two dependencies: openai and httpx.

The project, created by Kenneth Wolters, is positioned as a drop-in replacement for LiteLLM's core API. If you're already using LiteLLM, switching is as simple as changing your import statements from litellm to litelm. The API mirrors LiteLLM's function names, arguments, and response types, so the learning curve is minimal. This focus on simplicity is resonating with developers, as the project has already gained traction on Hacker News.

The Problem: LiteLLM's Growing Complexity

LiteLLM has become a standard tool for developers who need to interact with multiple LLM providers—OpenAI, Anthropic, Groq, Mistral, and many others—without rewriting their code for each API. It handles the translation between different message formats and routes requests to the correct endpoint based on a simple provider/model syntax. However, this convenience comes at a cost.

The full LiteLLM package is massive. It includes a proxy server for production deployments, caching layers, budgeting and cost tracking, token counting, image generation, audio processing, OCR, fine-tuning, agents, guardrails, and a scheduler. For a developer who just wants to make a few API calls from a script or a small application, this is overwhelming. It increases installation size, slows down imports, and introduces a larger attack surface and more potential points of failure.

Litelm addresses this by extracting only the call path—the essential functionality that makes LiteLLM useful in the first place. It deliberately omits the Router class for load balancing and fallbacks, the proxy server, and all the auxiliary features. This is a design philosophy that prioritizes simplicity and performance over feature completeness.

What Litelm Includes (and Excludes)

The project's README provides a clear comparison of what's in and what's out. Here's a breakdown of the core features that Litelm retains:

• Model routing: Using the provider/model syntax to direct requests to the correct endpoint.
• Message translation: Converting between the message formats of Anthropic, Bedrock, Cloudflare, Mistral, and others.
• Streaming: Support for streaming responses and a stream_chunk_builder utility.
• Tool use: Full support for function calling.
• Embeddings: Generating vector embeddings.
• Text completions: The classic completion endpoint.
• OpenAI Responses API: Support for the newer Responses API.
• Mock responses: For testing without real API calls.

On the other hand, it explicitly drops the following heavy components:

• Router: Load balancing, fallbacks, and retries.
• Proxy server: The production-grade gateway.
• Caching, budgeting, and cost tracking: Operational concerns.
• Token counting: A utility that adds dependencies.
• Image gen, audio, OCR, fine-tuning: Non-core modalities.
• Agents, guardrails, and scheduler: Higher-level abstractions.

This makes Litelm an ideal choice for developers building lightweight applications, scripts, or tools where the overhead of LiteLLM is disproportionate to the value it provides.

Provider Support and Compatibility

Despite its small size, Litelm routes to 19 providers using the "provider/model-name" syntax. This includes major players like OpenAI, Anthropic, Groq, Mistral, xAI, OpenRouter, and Azure, as well as local and self-hosted options like Ollama, vLLM, and LM Studio. For any OpenAI-compatible endpoint, you can also pass a custom api_base URL to connect to your own infrastructure.

The project's maintainer has provided an attestation regarding compatibility. As of September 11, 2026, the codebase was reviewed against LiteLLM's routing and formatting changes from commit 649eb2d through 9a715df2. The audit triaged 360 core-path commits and inspected upstream tests for relevant behavior. The result: 262 local tests passed (with 55 skipped), and all 45 available live provider tests and 10 DSPy smoke tests also passed. This gives a degree of confidence that Litelm's core functionality aligns with LiteLLM's, at least for the scoped surface area.

Getting Started: A Simple, Familiar API

One of the most appealing aspects of Litelm is its simplicity. Installation is straightforward using pip, and the API is immediately familiar to anyone who has used LiteLLM.

import litelm

# Basic completion
response = litelm.completion("openai/gpt-4o", messages=[{"role": "user", "content": "Hello!"}])
print(response.choices[0].message.content)

# Streaming
for chunk in litelm.completion("groq/llama-3.1-70b-versatile", messages=[...], stream=True):
    print(chunk.choices[0].delta.content or "", end="")

# Embeddings
response = litelm.embedding("openai/text-embedding-3-small", input="hello world")

Every function has an async variant (acompletion, aembedding, etc.), and error handling is consistent with a custom exception hierarchy that includes ContextWindowExceededError, RateLimitError, and AuthenticationError. This makes it easy to build robust applications with proper error handling.

Community Response and Significance

Litelm has already caught the attention of the developer community. It was shared on Hacker News, where it garnered early points and discussion. The sentiment is captured in a tweet by Satoshi Nagayasu, who noted "こういうところあるよね" ("This is a thing, isn't it"), highlighting the common desire for leaner tools in the AI ecosystem. The project is currently in alpha status, but its 14 GitHub stars and active development suggest a growing interest.

The significance of Litelm goes beyond just being a smaller library. It represents a broader trend in the AI development space: the push for modularity and minimalism. As AI models become more integrated into everyday software, developers are increasingly looking for tools that do one thing well without dragging in the entire ecosystem. Litelm is a response to that need, offering a focused, performant alternative to a popular but heavyweight tool.

Development Transparency and Future Outlook

Interestingly, the project is transparent about its development process. The README states that it is "human-directed, AI-assisted software," with much of the code written using Claude Code (Claude Opus 4.6/4.7) and, from May 14, 2026, onward, through Pi using GPT-5.5. This transparency about AI involvement is becoming more common but is still noteworthy in an open-source context.

Looking ahead, Litelm's future will depend on its ability to keep pace with LiteLLM's changes and the broader LLM ecosystem. The maintainer has already demonstrated a commitment to compatibility, and the project's clear scope should make it easier to maintain than its larger counterpart. While it may not be suitable for large-scale production deployments that require a proxy and advanced routing, it is an excellent choice for developers who value simplicity and speed.

In a world where AI tools are often criticized for their bloat, Litelm is a breath of fresh air. It proves that you can have the power of multi-provider LLM routing without the baggage.