# How to Add a Custom LLM Provider to LiteLLM: Complete Implementation Guide

> Easily add a custom LLM provider to LiteLLM by subclassing CustomLLM and registering your provider. This guide shows you how to seamlessly integrate any proprietary or third-party language model.

- Repository: [Berri AI/litellm](https://github.com/BerriAI/litellm)
- Tags: how-to-guide
- Published: 2026-03-26

---

**LiteLLM enables you to integrate any proprietary or third-party language model by subclassing `CustomLLM`, registering your provider in `custom_provider_map`, and letting the built-in router handle request dispatching.**

The BerriAI/litellm repository provides a pluggable architecture that treats every supported model as a provider implementing a common interface. Adding a custom LLM provider to LiteLLM requires implementing three core components: a handler class, a registry entry, and router integration. This guide walks through the complete implementation using the actual source code structure from the repository.

## Understanding the Custom Provider Architecture

LiteLLM's extensibility relies on three coordinated components that bridge your custom logic with the library's unified API:

- **Handler class** – A concrete implementation of the `CustomLLM` abstract base that defines how your model handles `completion`, `acompletion`, `streaming`, and `embedding` requests. The base class lives in [`litellm/llms/custom_llm.py`](https://github.com/BerriAI/litellm/blob/main/litellm/llms/custom_llm.py).

- **Provider registry** – The global `litellm.custom_provider_map` list stores `CustomLLMItem` entries that map provider names to handler instances. This registry is defined in [`litellm/__init__.py`](https://github.com/BerriAI/litellm/blob/main/litellm/__init__.py) alongside the internal `_custom_providers` tracker.

- **Router integration** – The core request handler in [`litellm/main.py`](https://github.com/BerriAI/litellm/blob/main/litellm/main.py) detects custom providers via the `elif custom_llm_provider in litellm._custom_providers:` block and dispatches calls using `custom_chat_llm_router`.

The registration helper `custom_llm_setup()` in [`litellm/utils.py`](https://github.com/BerriAI/litellm/blob/main/litellm/utils.py) synchronizes the registry with LiteLLM's internal provider lists at runtime.

## Step 1: Create a Custom LLM Handler

Subclass `CustomLLM` from [`litellm/llms/custom_llm.py`](https://github.com/BerriAI/litellm/blob/main/litellm/llms/custom_llm.py) to implement the methods that communicate with your service. You must define synchronous and asynchronous variants for completion and streaming operations.

```python

# my_custom_llm.py

import litellm
from litellm import CustomLLM
from litellm.types.utils import ModelResponse

class MyCustomLLM(CustomLLM):
    def completion(self, *args, **kwargs) -> ModelResponse:
        # Implement your external service call here

        return litellm.completion(
            model="gpt-3.5-turbo",
            messages=[{"role": "user", "content": "Hello"}],
            mock_response="Hi from my custom LLM!",
        )  # type: ignore

    async def acompletion(self, *args, **kwargs) -> ModelResponse:
        # Async implementation using httpx.AsyncClient or similar

        return litellm.completion(
            model="gpt-3.5-turbo",
            messages=[{"role": "user", "content": "Hello"}],
            mock_response="Hi async!",
        )  # type: ignore

    
    def streaming(self, *args, **kwargs):
        # Optional: implement for streaming support

        pass
    
    async def astreaming(self, *args, **kwargs):
        # Optional: implement for async streaming support

        pass

```

The `CustomLLM` base class defines the interface contract that LiteLLM expects. Return values must conform to `ModelResponse` or generator types for streaming operations.

## Step 2: Register Your Provider

Insert your handler into `litellm.custom_provider_map` before making any completion calls. Each entry follows the `CustomLLMItem` structure defined in [`litellm/types/llms/custom_llm.py`](https://github.com/BerriAI/litellm/blob/main/litellm/types/llms/custom_llm.py).

```python

# registration.py

import litellm
from my_custom_llm import MyCustomLLM
from litellm.types.llms.custom_llm import CustomLLMItem

# Create the registry entry

my_provider: CustomLLMItem = {
    "provider": "my-custom-llm",      # Name used in model= parameter

    "custom_handler": MyCustomLLM(),
    "model_group": "my_custom_group", # Optional grouping metadata

    "is_proxy_model": False,          # True if proxying a known provider

}

# Append to global registry

litellm.custom_provider_map.append(my_provider)

# Synchronize with LiteLLM's internal provider lists

litellm.custom_llm_setup()

```

Execute this registration code during application startup or inside a plugin package's [`__init__.py`](https://github.com/BerriAI/litellm/blob/main/__init__.py). The `custom_llm_setup()` function populates `litellm.provider_list` and the `_custom_providers` internal tracker, making your provider discoverable to the router.

## Step 3: Route Requests to Your Provider

When you call `litellm.completion(model="my-custom-llm", ...)`, the logic in [`litellm/main.py`](https://github.com/BerriAI/litellm/blob/main/litellm/main.py) detects the provider name in `litellm._custom_providers` and routes the request through `custom_chat_llm_router`. This dispatcher selects the appropriate method—`completion` vs `acompletion`, streaming vs non-streaming—based on your call parameters.

LiteLLM automatically handles:

- **Sync vs async detection** – Routes to `completion` or `acompletion` based on the calling context
- **Streaming protocol** – Invokes `streaming` or `astreaming` when `stream=True` is passed
- **Response wrapping** – Converts your return values into standard LiteLLM response objects

## Complete Working Example

Here is a complete implementation that integrates a fictional external API:

```python

# custom_integration.py

import litellm
from litellm import CustomLLM
from litellm.types.llms.custom_llm import CustomLLMItem
from litellm.types.utils import ModelResponse
import httpx

class ExternalAPIHandler(CustomLLM):
    def completion(self, model: str, messages: list, api_key: str, **kwargs) -> ModelResponse:
        # Replace with actual external API call

        response_text = f"Processed {len(messages)} messages via external API"
        
        return ModelResponse(
            choices=[{
                "message": {"content": response_text, "role": "assistant"},
                "index": 0,
                "finish_reason": "stop"
            }],
            model=model
        )
    
    async def acompletion(self, model: str, messages: list, api_key: str, **kwargs) -> ModelResponse:
        async with httpx.AsyncClient() as client:
            # Async external API implementation

            return self.completion(model, messages, api_key, **kwargs)

# Registration

handler = ExternalAPIHandler()
litellm.custom_provider_map.append({
    "provider": "external-api",
    "custom_handler": handler,
    "model_group": "external_models",
    "is_proxy_model": False
})
litellm.custom_llm_setup()

# Usage

response = litellm.completion(
    model="external-api",
    messages=[{"role": "user", "content": "Analyze this data"}]
)
print(response.choices[0].message.content)

```

## Summary

- **Subclass `CustomLLM`** from [`litellm/llms/custom_llm.py`](https://github.com/BerriAI/litellm/blob/main/litellm/llms/custom_llm.py) to implement `completion`, `acompletion`, and optional streaming methods for your service.
- **Register via `custom_provider_map`** by appending a `CustomLLMItem` dictionary that maps your provider name to the handler instance.
- **Call `custom_llm_setup()`** from [`litellm/utils.py`](https://github.com/BerriAI/litellm/blob/main/litellm/utils.py) to synchronize the registry with LiteLLM's internal provider lists before making requests.
- **Route automatically** through [`litellm/main.py`](https://github.com/BerriAI/litellm/blob/main/litellm/main.py) by passing your registered provider name to the `model` parameter in `litellm.completion()` or `litellm.acompletion()`.

## Frequently Asked Questions

### What methods must I implement in a custom LLM provider?

You must implement at least `completion` and `acompletion` to handle synchronous and asynchronous requests. For streaming support, implement `streaming` and `astreaming`. The `CustomLLM` base class in [`litellm/llms/custom_llm.py`](https://github.com/BerriAI/litellm/blob/main/litellm/llms/custom_llm.py) defines these method signatures and provides the interface contract that LiteLLM expects.

### Where does LiteLLM store the list of available custom providers?

LiteLLM maintains two key structures in [`litellm/__init__.py`](https://github.com/BerriAI/litellm/blob/main/litellm/__init__.py): `custom_provider_map` (the public registry of `CustomLLMItem` entries) and `_custom_providers` (the internal list of provider names). The `custom_llm_setup()` function in [`litellm/utils.py`](https://github.com/BerriAI/litellm/blob/main/litellm/utils.py) populates these structures at runtime.

### Can I register multiple custom providers in the same application?

Yes. Append multiple `CustomLLMItem` dictionaries to `litellm.custom_provider_map` with unique provider names, then call `litellm.custom_llm_setup()` once after all registrations. Each provider operates independently, allowing you to route different models to different external services using the same LiteLLM API.

### How does LiteLLM decide which method to call on my custom handler?

The routing logic in [`litellm/main.py`](https://github.com/BerriAI/litellm/blob/main/litellm/main.py) checks if the `custom_llm_provider` parameter exists in `litellm._custom_providers`. If found, it uses `custom_chat_llm_router` to select between `completion`, `acompletion`, `streaming`, or `astreaming` based on whether the call uses `async` and whether `stream=True` is passed.