# How to Set Up LiteLLM Provider with Custom Model Endpoints in OpenViking

> Easily set up LiteLLM provider with custom model endpoints in OpenViking. Connect to self-hosted or private LLM servers by configuring api_base URL or providers section within VLM config.

- Repository: [Volcengine/OpenViking](https://github.com/volcengine/OpenViking)
- Tags: how-to-guide
- Published: 2026-03-08

---

**You configure a custom endpoint in OpenViking by passing the `api_base` URL to `LiteLLMProvider` or declaring it in the `providers` section of your VLM configuration, which automatically routes requests to your self-hosted or private LLM server.**

OpenViking uses **LiteLLM** as a unified wrapper for multiple LLM services, enabling seamless integration with both commercial APIs and private endpoints. The `LiteLLMProvider` class in the `volcengine/OpenViking` repository handles the heavy lifting by forwarding your custom `api_base` directly to LiteLLM while managing environment variables through the provider registry. Whether you are connecting to a self-hosted Moonshot instance or a private OpenAI-compatible server, you only need to supply the endpoint URL and optional authentication credentials.

## Understanding the LiteLLM Provider Architecture

The integration relies on three core components that work together to resolve and route requests to custom endpoints.

**LiteLLMProvider** ([`bot/vikingbot/providers/litellm_provider.py`](https://github.com/volcengine/OpenViking/blob/main/bot/vikingbot/providers/litellm_provider.py)) implements the LLM interface and manages the actual API calls. It stores your `api_base` during initialization and forwards it unchanged to LiteLLM during the `chat` method execution.

**Provider Registry** ([`bot/vikingbot/providers/registry.py`](https://github.com/volcengine/OpenViking/blob/main/bot/vikingbot/providers/registry.py)) maintains `ProviderSpec` definitions for every supported service. The `find_gateway` function detects gateway and local providers based on your `provider_name`, API key prefix, or `api_base` keyword, ensuring the correct environment variables are set.

**VLM Configuration** ([`openviking_cli/utils/config/vlm_config.py`](https://github.com/volcengine/OpenViking/blob/main/openviking_cli/utils/config/vlm_config.py)) provides the Pydantic model that aggregates settings from your config files. The `_build_vlm_config_dict` method converts the `providers` mapping into the dictionary passed to `LiteLLMProvider`, including your custom `api_base` and `api_key`.

## Configuring Custom Endpoints via VLMConfig

The simplest way to set up a custom model endpoint is through the YAML configuration file used by the OpenViking CLI.

Add your endpoint details under the `providers` section:

```yaml

# config.yaml

vlm:
  model: "kimi-k2.5"
  provider: "moonshot"
  providers:
    moonshot:
      api_key: "my-moonshot-key"
      api_base: "https://my.private.moonshot/api/v1"

```

When the CLI loads this configuration, `VLMConfig._build_vlm_config_dict` (lines 39-44) constructs the runtime dictionary:

```python
{
    "model": "kimi-k2.5",
    "temperature": 0.0,
    "max_retries": 2,
    "provider": "moonshot",
    "api_key": "my-moonshot-key",
    "api_base": "https://my.private.moonshot/api/v1",
}

```

`VLMFactory.create` then instantiates `LiteLLMProvider` with these values, automatically routing requests to your private endpoint instead of the default Moonshot API.

## Programmatic Setup with LiteLLMProvider

For dynamic configurations or custom applications, instantiate `LiteLLMProvider` directly with your endpoint parameters:

```python
from bot.vikingbot.providers.litellm_provider import LiteLLMProvider

# Configure for a self-hosted OpenAI-compatible server

custom_provider = LiteLLMProvider(
    api_key="my-selfhosted-key",
    api_base="http://localhost:8000/v1",
    provider_name="openai",
    default_model="gpt-4o-mini",
)

# Execute chat completion

response = await custom_provider.chat(
    messages=[{"role": "user", "content": "What is the weather today?"}],
    model="gpt-4o-mini",
)
print(response.content)

```

**How this works:**

1. **Initialization** (`__init__`, lines 26-33): Stores `api_base` as `self.api_base` and calls `_setup_env` to configure environment variables.

2. **Gateway Detection** (`find_gateway` in [`registry.py`](https://github.com/volcengine/OpenViking/blob/main/registry.py), lines 34-45): Uses your `provider_name` or `api_base` keyword to select the correct `ProviderSpec`, ensuring LiteLLM receives the proper routing prefix.

3. **Request Forwarding** (`chat` method, lines 46-49): Injects `kwargs["api_base"] = self.api_base` into the LiteLLM call, overriding the default base URL with your custom endpoint.

## Adding Extra Headers for Custom Authentication

Some private endpoints require additional headers beyond standard API keys. Pass these through the `extra_headers` parameter:

```python
custom_provider = LiteLLMProvider(
    api_key="my-key",
    api_base="https://my.service/v1",
    extra_headers={"APP-Code": "my-app-code", "X-Custom-Auth": "token"},
    default_model="my-model",
)

response = await custom_provider.chat(
    messages=[{"role": "user", "content": "Explain the term 'LLM'"}],
    model="my-model",
)

```

The provider injects these headers into the LiteLLM request via `kwargs["extra_headers"] = self.extra_headers` (lines 50-53 in [`litellm_provider.py`](https://github.com/volcengine/OpenViking/blob/main/litellm_provider.py)).

## How the Registry Resolves Custom Endpoints

The provider registry ([`bot/vikingbot/providers/registry.py`](https://github.com/volcengine/OpenViking/blob/main/bot/vikingbot/providers/registry.py)) contains `ProviderSpec` objects that define environment variable mappings and LiteLLM prefixes for each service. When you specify a custom `api_base`, the `find_gateway` function checks detection rules including:

- **Provider name matching** (e.g., `"moonshot"`, `"openai"`)
- **API base keyword detection** (e.g., URLs containing `"moonshot"` trigger the Moonshot spec)

This automatic resolution ensures that the correct environment variables (such as `MOONSHOT_API_KEY` or `OPENAI_API_KEY`) are exported via `_setup_env` (lines 57-78) before LiteLLM executes the request.

## Summary

- **Declare endpoints in config**: Add `api_base` and `api_key` to the `providers` section of `VLMConfig` for CLI-based usage.
- **Instantiate programmatically**: Pass `api_base` directly to `LiteLLMProvider` for dynamic or embedded applications.
- **Automatic resolution**: The provider registry ([`registry.py`](https://github.com/volcengine/OpenViking/blob/main/registry.py)) detects your endpoint type and sets required environment variables via `find_gateway` and `_setup_env`.
- **Header support**: Use `extra_headers` for proprietary authentication schemes required by private endpoints.
- **Direct forwarding**: The `chat` method passes `api_base` unchanged to LiteLLM, ensuring requests reach your custom URL.

## Frequently Asked Questions

### How does OpenViking route requests to my custom API endpoint?

OpenViking routes requests by storing your custom URL in `LiteLLMProvider.api_base` and forwarding it to LiteLLM as the `api_base` parameter during the `chat` call (lines 46-48 in [`litellm_provider.py`](https://github.com/volcengine/OpenViking/blob/main/litellm_provider.py)). LiteLLM then constructs the full request URL using your endpoint instead of the default provider URL.

### Can I use custom endpoints with any LLM provider in the registry?

Yes. The `find_gateway` function in [`registry.py`](https://github.com/volcengine/OpenViking/blob/main/registry.py) (lines 34-45) supports custom endpoints for any defined `ProviderSpec`. You can override the default `api_base` for OpenAI, Moonshot, Anthropic, or any other registered provider by specifying the `api_base` URL in your configuration or constructor.

### Where do I store API keys for custom endpoints?

Store API keys in the `providers` section of your `VLMConfig` YAML file under the specific provider key (e.g., `providers.moonshot.api_key`), or pass them directly to the `LiteLLMProvider` constructor as the `api_key` parameter. The `_setup_env` method (lines 57-78) automatically maps these to the correct environment variables expected by LiteLLM.

### How do I debug connection issues with custom endpoints?

Enable LiteLLM debugging by removing the `litellm.suppress_debug_info = True` line in `LiteLLMProvider.__init__` (lines 52-55), or set `litellm.set_verbose=True` in your code. Verify that your `api_base` includes the correct API version path (typically `/v1` for OpenAI-compatible endpoints) and that the `provider_name` matches a valid entry in [`registry.py`](https://github.com/volcengine/OpenViking/blob/main/registry.py) to ensure proper environment variable configuration.