# How Lemon AI Integrates with Ollama and VLLM for Local LLM Support

> Learn how Lemon AI integrates with Ollama and VLLM for local LLM support. Discover its generic OpenAI client for seamless local endpoint integration without API keys.

- Repository: [hexdocom/lemonai](https://github.com/hexdocom/lemonai)
- Tags: deep-dive
- Published: 2026-03-03

---

**Lemon AI treats Ollama and VLLM as PRIVATE channel providers, routing requests through a generic OpenAI-compatible client that forwards HTTP calls to local endpoints like `http://localhost:11434/v1` without requiring API keys.**

The `hexdocom/lemonai` repository implements a unified provider architecture that enables seamless **Lemon AI Ollama VLLM integration** for self-hosted language models. By abstracting every LLM behind a standardized provider string and channel selection logic, the system eliminates the complexity of managing different local inference engines separately. This approach allows developers to switch between commercial APIs and local instances without changing application code.

## Provider String Architecture and Model Resolution

Every LLM request begins with a provider string formatted as `provider#<platform_name>#<model_name>`. In [`src/utils/llm.js`](https://github.com/hexdocom/lemonai/blob/main/src/utils/llm.js), line 27 constructs this identifier:

```javascript
const model = `provider#${model_info.platform_name}#${model_info.model_name}`;

```

The `createLLMInstance` function then passes this string to the factory in [`src/completion/llm.one.js`](https://github.com/hexdocom/lemonai/blob/main/src/completion/llm.one.js). Lines 58-66 parse the identifier into three components: **channel**, **service**, and **model**. For platforms that do not require commercial API authentication, the system selects the **PRIVATE** channel, triggering a configuration-based instantiation rather than a vendor-specific implementation.

## PRIVATE Channel Configuration for Local Models

When the channel resolves to **PRIVATE**, Lemon AI bypasses proprietary SDKs in favor of generic OpenAI-compatible HTTP clients. The `resolveServiceConfig` function in [`src/completion/resolveServiceConfig.js`](https://github.com/hexdocom/lemonai/blob/main/src/completion/resolveServiceConfig.js) (lines 3-9) retrieves platform settings from the JSON data store, returning an object containing `api_url` and optional `api_key` fields.

This configuration drives the `ConfigLLM` class defined in [`src/completion/llm.config.js`](https://github.com/hexdocom/lemonai/blob/main/src/completion/llm.config.js) (lines 5-12), which extends the base functionality from [`src/completion/llm.base.js`](https://github.com/hexdocom/lemonai/blob/main/src/completion/llm.base.js). Because both Ollama and VLLM expose endpoints compatible with OpenAI's `/v1/chat/completions` schema, the same `ConfigLLM` implementation handles request serialization and token streaming for both platforms.

## Ollama Integration Implementation

Ollama ships with a predefined entry in [`public/default_data/default_platform.json`](https://github.com/hexdocom/lemonai/blob/main/public/default_data/default_platform.json) (lines 168-175), configuring the default `api_url` as `http://localhost:11434/v1` with an empty API key. The UI component in [`frontend/src/view/setting/model.vue`](https://github.com/hexdocom/lemonai/blob/main/frontend/src/view/setting/model.vue) (lines 84-86) detects this platform name to hide the API-key input field, streamlining the setup experience.

When executing a chat completion, [`src/completion/llm.base.js`](https://github.com/hexdocom/lemonai/blob/main/src/completion/llm.base.js) (lines 75-104) sends a POST request to the configured endpoint using standard OpenAI message formatting. This allows Ollama's native API compatibility layer to handle model inference without custom adapters.

## VLLM Integration Setup

Unlike Ollama, VLLM requires manual platform registration but uses the same underlying infrastructure. Users add a platform entry with `provider_type: "OpenAI"` and specify the VLLM server URL (typically `http://localhost:8000/v1`) in the `api_url` field.

The system treats this identically to the Ollama flow: [`src/completion/llm.one.js`](https://github.com/hexdocom/lemonai/blob/main/src/completion/llm.one.js) routes it through the PRIVATE channel, `resolveServiceConfig` loads the endpoint configuration, and `ConfigLLM` manages the HTTP communication. This design means VLLM support requires no additional code changes beyond the platform configuration JSON.

## Practical Code Examples

To invoke an Ollama model from server code, use the high-level helper in [`src/utils/llm.js`](https://github.com/hexdocom/lemonai/blob/main/src/utils/llm.js):

```javascript
const response = await call(
  "Explain the difference between LLM and AGI",
  conversationId,
  "assistant",
  { temperature: 0.7, stream: true }
);

```

This internally constructs a `provider#Ollama#llama2` identifier and routes the request through the PRIVATE channel pipeline.

For direct instantiation bypassing the factory, import the generic client:

```javascript
const ConfigLLM = require('@src/completion/llm.config');

const ollama = new ConfigLLM(
  { url: 'http://localhost:11434/v1', model: 'llama2', api_key: '' },
  chunk => process.stdout.write(chunk)
);

ollama.completion('Write a short poem about AI', {}, { temperature: 0.5 })
  .then(console.log)
  .catch(console.error);

```

To register VLLM via configuration payload:

```json
{
  "name": "VLLM",
  "provider_type": "OpenAI",
  "api_url": "http://localhost:8000/v1",
  "api_key": "",
  "is_enabled": "True"
}

```

## Summary

- Lemon AI uses a **provider string** format (`provider#platform#model`) to abstract all LLM integrations in [`src/utils/llm.js`](https://github.com/hexdocom/lemonai/blob/main/src/utils/llm.js).
- **PRIVATE** channel selection in [`src/completion/llm.one.js`](https://github.com/hexdocom/lemonai/blob/main/src/completion/llm.one.js) triggers generic OpenAI-compatible clients for local models.
- The `ConfigLLM` class in [`src/completion/llm.config.js`](https://github.com/hexdocom/lemonai/blob/main/src/completion/llm.config.js) handles HTTP requests to Ollama and VLLM endpoints using standard `/v1/chat/completions` schemas.
- Ollama comes pre-configured in [`public/default_data/default_platform.json`](https://github.com/hexdocom/lemonai/blob/main/public/default_data/default_platform.json) with UI support hiding API-key fields in [`frontend/src/view/setting/model.vue`](https://github.com/hexdocom/lemonai/blob/main/frontend/src/view/setting/model.vue).
- VLLM integrates through the same code path by registering a custom platform with a local `api_url`.

## Frequently Asked Questions

### Does Lemon AI require an API key for Ollama or VLLM?

No. Local LLM integrations use the PRIVATE channel, which expects an empty or optional `api_key` field. The UI automatically hides the API-key input when Ollama is selected, and the `ConfigLLM` class does not inject authentication headers when the key is absent.

### What file handles the routing decision between commercial and local LLMs?

[`src/completion/llm.one.js`](https://github.com/hexdocom/lemonai/blob/main/src/completion/llm.one.js) contains the factory logic that parses the provider string and selects the channel. Lines 81-90 specifically check for `channel === CHANNEL.PRIVATE` to instantiate the generic `ConfigLLM` instead of vendor-specific implementations.

### Can I use the same code to call both Ollama and OpenAI models?

Yes. The `call()` function in [`src/utils/llm.js`](https://github.com/hexdocom/lemonai/blob/main/src/utils/llm.js) accepts any provider string, whether pointing to OpenAI's commercial API or a local Ollama instance. The architecture ensures that application code remains agnostic to the underlying inference engine, with differences confined to the platform configuration and channel selection logic.

### How do I add a custom VLLM server to Lemon AI?

Create a platform entry in your configuration store with `provider_type` set to `"OpenAI"` and your server's URL in the `api_url` field (e.g., `http://localhost:8000/v1`). The `resolveServiceConfig` function will load this entry, and [`llm.one.js`](https://github.com/hexdocom/lemonai/blob/main/llm.one.js) will route requests through the PRIVATE channel to your VLLM endpoint using the standard OpenAI HTTP format.