# How to Configure Vane's LLM Providers and Models: A Complete Guide

> Learn how to configure Vane's LLM providers and models with this complete guide. Master Vane's three-layer architecture for seamless integration and model loading.

- Repository: [Kushagra Srivastava/Vane](https://github.com/ItzCrazyKns/Vane)
- Tags: how-to-guide
- Published: 2026-03-11

---

**Vane configures LLM providers through a three-layer architecture where you declare configuration fields in TypeScript, persist instances via REST API or environment variables to [`config.json`](https://github.com/ItzCrazyKns/Vane/blob/main/config.json), and load models through the `ModelRegistry` class.**

Vane is an open-source AI search engine that abstracts every LLM service as a configurable provider. Whether you need to connect OpenAI, Ollama, or Gemini, understanding how to configure Vane's LLM providers and models is essential for customizing your search experience.

## Understanding Vane's Provider Architecture

Vane treats every LLM service as a *provider* supplying two model types:

* **Chat models** – used for generating text (e.g., `gpt-4`, `gemini-1.5-pro`).
* **Embedding models** – used for vector-embedding generation (e.g., `text-embedding-3-small`).

The configuration flow consists of three layers:

| Layer | Responsibility | Core files |
|-------|----------------|------------|
| **UI / Environment** | Declares which fields a provider needs (API key, base URL, etc.) and maps them to environment variables. | [`src/lib/config/types.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/config/types.ts), [`src/lib/config/index.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/config/index.ts), [`src/lib/models/providers/index.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/providers/index.ts) |
| **Runtime Registry** | Reads the persisted **config.json**, builds concrete provider instances, and exposes the list of active providers + their model catalogs. | [`src/lib/models/registry.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/registry.ts), [`src/lib/models/base/provider.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/base/provider.ts), [`src/lib/config/serverRegistry.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/config/serverRegistry.ts) |
| **API / Consumer** | Public HTTP endpoints let the front-end (or scripts) add, update, or delete providers; the app then loads the requested chat or embedding model. | [`src/app/api/providers/route.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/app/api/providers/route.ts), `src/app/api/providers/[id]/models/route.ts` |

## Declaring Provider Configuration Fields

Each provider ships a static `getProviderConfigFields()` method that returns an array of **UIConfigField** definitions. For OpenAI, this lives in [`src/lib/models/providers/openai/index.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/providers/openai/index.ts):

```typescript
const providerConfigFields: UIConfigField[] = [
  {
    type: 'password',
    name: 'API Key',
    key: 'apiKey',
    description: 'Your OpenAI API key',
    required: true,
    env: 'OPENAI_API_KEY',          // ← linked to an env var
    scope: 'server',
  },
  {
    type: 'string',
    name: 'Base URL',
    key: 'baseURL',
    description: 'The base URL for the OpenAI API',
    required: true,
    default: 'https://api.openai.com/v1',
    env: 'OPENAI_BASE_URL',
    scope: 'server',
  },
];

```

When Vane boots, `ConfigManager` in [`src/lib/config/index.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/config/index.ts) reads all providers via `getModelProvidersUIConfigSection()` and automatically builds UI sections for the Settings page. The UI fields are stored in `ConfigManager.uiConfigSections.modelProviders`, later used when populating the *Add Provider* dialog.

## Persisting Provider Instances

Vane stores providers in a JSON file ([`data/config.json`](https://github.com/ItzCrazyKns/Vane/blob/main/data/config.json), default path built in `ConfigManager.configPath`). The **model provider schema** lives in [`src/lib/config/types.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/config/types.ts):

```typescript
type ConfigModelProvider = {
  id: string;
  name: string;
  type: string;           // e.g. "openai", "ollama", …
  chatModels: Model[];
  embeddingModels: Model[];
  config: { [key: string]: any };
  hash: string;
};

```

When environment variables satisfy every required field, `ConfigManager.initializeFromEnv()` auto-creates a provider entry (hash-deduped) and writes it to the file.

**Manual addition** is done via the HTTP **POST /api/providers** endpoint in [`src/app/api/providers/route.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/app/api/providers/route.ts). The request body must contain:

```json
{
  "type": "openai",
  "name": "My OpenAI",
  "config": {
    "apiKey": "sk-xxxxxxxxxxxx",
    "baseURL": "https://api.openai.com/v1"
  }
}

```

The route calls `ModelRegistry.addProvider()`, which:
1. Persists the provider via `configManager.addModelProvider()`.
2. Instantiates the concrete provider (`createProviderInstance` from [`src/lib/models/base/provider.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/base/provider.ts)).
3. Queries the provider for its model list and returns the enriched provider object.

## Building the Active Provider Registry

`ModelRegistry` in [`src/lib/models/registry.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/registry.ts) is the runtime façade:

```typescript
export default class ModelRegistry {
  activeProviders = [];   // populated in the constructor
  constructor() {
    this.initializeActiveProviders();
  }
  // … (methods for getActiveProviders, loadChatModel, loadEmbeddingModel)
}

```

During construction it reads **configured providers** (`getConfiguredModelProviders()` from [`src/lib/config/serverRegistry.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/config/serverRegistry.ts)) and creates an instance for each:

```typescript
const provider = providers[p.type]; // providers map from src/lib/models/providers/index.ts
this.activeProviders.push({
  ...p,
  provider: createProviderInstance(provider, p.id, p.name, p.config),
});

```

The registry exposes:
* `getActiveProviders()` – returns an array of providers with their **chatModels** and **embeddingModels** (used by the UI to display available models).
* `loadChatModel(providerId, modelName)` – returns a concrete `BaseLLM` (e.g., `OpenAILLM`).
* `loadEmbeddingModel(providerId, modelName)` – returns a concrete `BaseEmbedding`.

## Loading Specific Chat and Embedding Models

### Chat model (LLM)

```typescript
import ModelRegistry from '@/lib/models/registry';

async function getChatModel() {
  const registry = new ModelRegistry();
  // Suppose we want "gpt-4" from the OpenAI provider we added earlier
  const llm = await registry.loadChatModel('provider-uuid-here', 'gpt-4');
  // `llm` is an instance of OpenAILLM (src/lib/models/providers/openai/openaiLLM.ts)
  const response = await llm.generateText({
    messages: [{ role: 'user', content: 'Explain quantum entanglement' }],
  });
  console.log(response.content);
}

```

`OpenAILLM` implements `generateText`, `streamText`, `generateObject`, etc., forwarding the call to the official OpenAI SDK.

### Embedding model

```typescript
async function getEmbeddingModel() {
  const registry = new ModelRegistry();
  const embedder = await registry.loadEmbeddingModel(
    'provider-uuid-here',
    'text-embedding-3-small',
  );
  const vectors = await embedder.embedText({ text: 'Hello world' });
  console.log(vectors);
}

```

`OpenAIEmbedding` (found in [`src/lib/models/providers/openai/openaiEmbedding.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/providers/openai/openaiEmbedding.ts)) wraps the `/embeddings` endpoint.

## Updating and Removing Providers

* **PATCH /api/providers/[id]** follows the same pattern as POST and triggers `ModelRegistry.updateProvider()`.
* **DELETE /api/providers/[id]** triggers `ModelRegistry.removeProvider()`, which updates [`config.json`](https://github.com/ItzCrazyKns/Vane/blob/main/config.json) and the in-memory list.

## Complete Configuration Example

Here is the full workflow from environment setup to generating a response:

```bash

# 1️⃣ Set env vars (optional – Vane reads them on startup)

export OPENAI_API_KEY=sk-xxxxxxxxxxxx
export OPENAI_BASE_URL=https://api.openai.com/v1

```

```typescript
// 2️⃣ Add the provider via the REST API (client-side)
await fetch('/api/providers', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    type: 'openai',
    name: 'My OpenAI',
    config: {
      apiKey: process.env.OPENAI_API_KEY,
      baseURL: process.env.OPENAI_BASE_URL,
    },
  }),
});

```

```typescript
// 3️⃣ Load the GPT-4 chat model and generate a response
import ModelRegistry from '@/lib/models/registry';

(async () => {
  const registry = new ModelRegistry();
  const providers = await registry.getActiveProviders(); // see UI list
  const openAi = providers.find(p => p.name === 'My OpenAI');
  if (!openAi) throw new Error('Provider not found');

  const llm = await registry.loadChatModel(openAi.id, 'gpt-4');
  const resp = await llm.generateText({
    messages: [{ role: 'user', content: 'Summarize the plot of *Inception*.' }],
  });
  console.log('LLM answer:', resp.content);
})();

```

The same pattern works for embedding models or for any other provider (Ollama, Gemini, Groq, etc.) – just replace the `"type"` and model keys.

## Summary

- **Three-layer architecture** separates UI configuration, runtime registry, and API consumption, making Vane extensible without core code changes.
- **Configuration fields** are declared via `getProviderConfigFields()` in provider implementations and automatically map to environment variables like `OPENAI_API_KEY`.
- **ModelRegistry** in [`src/lib/models/registry.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/registry.ts) manages active provider instances and exposes `loadChatModel()` and `loadEmbeddingModel()` for runtime model resolution.
- **Persistence** happens in [`data/config.json`](https://github.com/ItzCrazyKns/Vane/blob/main/data/config.json) via `ConfigManager`, with REST endpoints at `/api/providers` enabling dynamic provider management.

## Frequently Asked Questions

### How do I add a new LLM provider to Vane without restarting the application?

Send a **POST request to `/api/providers`** with the provider type and configuration object. The `ModelRegistry.addProvider()` method in [`src/lib/models/registry.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/registry.ts) dynamically instantiates the provider, queries its model list, and persists it to [`config.json`](https://github.com/ItzCrazyKns/Vane/blob/main/config.json) without requiring a server restart.

### What environment variables does Vane use for OpenAI configuration?

Vane recognizes `OPENAI_API_KEY` and `OPENAI_BASE_URL` as defined in the `env` property of the `UIConfigField` array within [`src/lib/models/providers/openai/index.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/providers/openai/index.ts). When these variables are present, `ConfigManager.initializeFromEnv()` automatically creates a provider entry on startup.

### How does Vane handle embedding models differently from chat models?

While both are managed through the same provider configuration in [`src/lib/config/types.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/config/types.ts), they are instantiated separately at runtime. The `ModelRegistry` class exposes `loadChatModel()` which returns a `BaseLLM` implementation (like `OpenAILLM`), while `loadEmbeddingModel()` returns a `BaseEmbedding` implementation (like `OpenAIEmbedding`), each wrapping distinct API endpoints.

### Where does Vane store provider configuration persistently?

Provider configurations are stored in [`data/config.json`](https://github.com/ItzCrazyKns/Vane/blob/main/data/config.json) (path defined in `ConfigManager.configPath`) as an array of `ConfigModelProvider` objects. This schema includes the provider `id`, `type`, `chatModels`, `embeddingModels`, and encrypted `config` values, enabling the `ModelRegistry` to rebuild active provider instances across server restarts.