How to Configure Vane's LLM Providers and Models: A Complete Guide

Vane configures LLM providers through a three-layer architecture where you declare configuration fields in TypeScript, persist instances via REST API or environment variables to config.json, and load models through the ModelRegistry class.

Vane is an open-source AI search engine that abstracts every LLM service as a configurable provider. Whether you need to connect OpenAI, Ollama, or Gemini, understanding how to configure Vane's LLM providers and models is essential for customizing your search experience.

Understanding Vane's Provider Architecture

Vane treats every LLM service as a provider supplying two model types:

• Chat models – used for generating text (e.g., gpt-4, gemini-1.5-pro).
• Embedding models – used for vector-embedding generation (e.g., text-embedding-3-small).

The configuration flow consists of three layers:

Layer	Responsibility	Core files
UI / Environment	Declares which fields a provider needs (API key, base URL, etc.) and maps them to environment variables.	src/lib/config/types.ts, src/lib/config/index.ts, src/lib/models/providers/index.ts
Runtime Registry	Reads the persisted config.json, builds concrete provider instances, and exposes the list of active providers + their model catalogs.	src/lib/models/registry.ts, src/lib/models/base/provider.ts, src/lib/config/serverRegistry.ts
API / Consumer	Public HTTP endpoints let the front-end (or scripts) add, update, or delete providers; the app then loads the requested chat or embedding model.	src/app/api/providers/route.ts, src/app/api/providers/[id]/models/route.ts

Declaring Provider Configuration Fields

Each provider ships a static getProviderConfigFields() method that returns an array of UIConfigField definitions. For OpenAI, this lives in src/lib/models/providers/openai/index.ts:

const providerConfigFields: UIConfigField[] = [
  {
    type: 'password',
    name: 'API Key',
    key: 'apiKey',
    description: 'Your OpenAI API key',
    required: true,
    env: 'OPENAI_API_KEY',          // ← linked to an env var
    scope: 'server',
  },
  {
    type: 'string',
    name: 'Base URL',
    key: 'baseURL',
    description: 'The base URL for the OpenAI API',
    required: true,
    default: 'https://api.openai.com/v1',
    env: 'OPENAI_BASE_URL',
    scope: 'server',
  },
];

When Vane boots, ConfigManager in src/lib/config/index.ts reads all providers via getModelProvidersUIConfigSection() and automatically builds UI sections for the Settings page. The UI fields are stored in ConfigManager.uiConfigSections.modelProviders, later used when populating the Add Provider dialog.

Persisting Provider Instances

Vane stores providers in a JSON file (data/config.json, default path built in ConfigManager.configPath). The model provider schema lives in src/lib/config/types.ts:

type ConfigModelProvider = {
  id: string;
  name: string;
  type: string;           // e.g. "openai", "ollama", …
  chatModels: Model[];
  embeddingModels: Model[];
  config: { [key: string]: any };
  hash: string;
};

When environment variables satisfy every required field, ConfigManager.initializeFromEnv() auto-creates a provider entry (hash-deduped) and writes it to the file.

Manual addition is done via the HTTP POST /api/providers endpoint in src/app/api/providers/route.ts. The request body must contain:

{
  "type": "openai",
  "name": "My OpenAI",
  "config": {
    "apiKey": "sk-xxxxxxxxxxxx",
    "baseURL": "https://api.openai.com/v1"
  }
}

The route calls ModelRegistry.addProvider(), which:

1. Persists the provider via configManager.addModelProvider().
2. Instantiates the concrete provider (createProviderInstance from src/lib/models/base/provider.ts).
3. Queries the provider for its model list and returns the enriched provider object.

Building the Active Provider Registry

ModelRegistry in src/lib/models/registry.ts is the runtime façade:

export default class ModelRegistry {
  activeProviders = [];   // populated in the constructor
  constructor() {
    this.initializeActiveProviders();
  }
  // … (methods for getActiveProviders, loadChatModel, loadEmbeddingModel)
}

During construction it reads configured providers (getConfiguredModelProviders() from src/lib/config/serverRegistry.ts) and creates an instance for each:

const provider = providers[p.type]; // providers map from src/lib/models/providers/index.ts
this.activeProviders.push({
  ...p,
  provider: createProviderInstance(provider, p.id, p.name, p.config),
});

The registry exposes:

• getActiveProviders() – returns an array of providers with their chatModels and embeddingModels (used by the UI to display available models).
• loadChatModel(providerId, modelName) – returns a concrete BaseLLM (e.g., OpenAILLM).
• loadEmbeddingModel(providerId, modelName) – returns a concrete BaseEmbedding.

Loading Specific Chat and Embedding Models

Chat model (LLM)

import ModelRegistry from '@/lib/models/registry';

async function getChatModel() {
  const registry = new ModelRegistry();
  // Suppose we want "gpt-4" from the OpenAI provider we added earlier
  const llm = await registry.loadChatModel('provider-uuid-here', 'gpt-4');
  // `llm` is an instance of OpenAILLM (src/lib/models/providers/openai/openaiLLM.ts)
  const response = await llm.generateText({
    messages: [{ role: 'user', content: 'Explain quantum entanglement' }],
  });
  console.log(response.content);
}

OpenAILLM implements generateText, streamText, generateObject, etc., forwarding the call to the official OpenAI SDK.

Embedding model

async function getEmbeddingModel() {
  const registry = new ModelRegistry();
  const embedder = await registry.loadEmbeddingModel(
    'provider-uuid-here',
    'text-embedding-3-small',
  );
  const vectors = await embedder.embedText({ text: 'Hello world' });
  console.log(vectors);
}

OpenAIEmbedding (found in src/lib/models/providers/openai/openaiEmbedding.ts) wraps the /embeddings endpoint.

Updating and Removing Providers

• PATCH /api/providers/[id] follows the same pattern as POST and triggers ModelRegistry.updateProvider().
• DELETE /api/providers/[id] triggers ModelRegistry.removeProvider(), which updates config.json and the in-memory list.

Complete Configuration Example

Here is the full workflow from environment setup to generating a response:

# 1️⃣ Set env vars (optional – Vane reads them on startup)

export OPENAI_API_KEY=sk-xxxxxxxxxxxx
export OPENAI_BASE_URL=https://api.openai.com/v1

// 2️⃣ Add the provider via the REST API (client-side)
await fetch('/api/providers', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    type: 'openai',
    name: 'My OpenAI',
    config: {
      apiKey: process.env.OPENAI_API_KEY,
      baseURL: process.env.OPENAI_BASE_URL,
    },
  }),
});

// 3️⃣ Load the GPT-4 chat model and generate a response
import ModelRegistry from '@/lib/models/registry';

(async () => {
  const registry = new ModelRegistry();
  const providers = await registry.getActiveProviders(); // see UI list
  const openAi = providers.find(p => p.name === 'My OpenAI');
  if (!openAi) throw new Error('Provider not found');

  const llm = await registry.loadChatModel(openAi.id, 'gpt-4');
  const resp = await llm.generateText({
    messages: [{ role: 'user', content: 'Summarize the plot of *Inception*.' }],
  });
  console.log('LLM answer:', resp.content);
})();

The same pattern works for embedding models or for any other provider (Ollama, Gemini, Groq, etc.) – just replace the "type" and model keys.

Summary

• Three-layer architecture separates UI configuration, runtime registry, and API consumption, making Vane extensible without core code changes.
• Configuration fields are declared via getProviderConfigFields() in provider implementations and automatically map to environment variables like OPENAI_API_KEY.
• ModelRegistry in src/lib/models/registry.ts manages active provider instances and exposes loadChatModel() and loadEmbeddingModel() for runtime model resolution.
• Persistence happens in data/config.json via ConfigManager, with REST endpoints at /api/providers enabling dynamic provider management.

Frequently Asked Questions

How do I add a new LLM provider to Vane without restarting the application?

Send a POST request to /api/providers with the provider type and configuration object. The ModelRegistry.addProvider() method in src/lib/models/registry.ts dynamically instantiates the provider, queries its model list, and persists it to config.json without requiring a server restart.

What environment variables does Vane use for OpenAI configuration?

Vane recognizes OPENAI_API_KEY and OPENAI_BASE_URL as defined in the env property of the UIConfigField array within src/lib/models/providers/openai/index.ts. When these variables are present, ConfigManager.initializeFromEnv() automatically creates a provider entry on startup.

How does Vane handle embedding models differently from chat models?

While both are managed through the same provider configuration in src/lib/config/types.ts, they are instantiated separately at runtime. The ModelRegistry class exposes loadChatModel() which returns a BaseLLM implementation (like OpenAILLM), while loadEmbeddingModel() returns a BaseEmbedding implementation (like OpenAIEmbedding), each wrapping distinct API endpoints.

Where does Vane store provider configuration persistently?

Provider configurations are stored in data/config.json (path defined in ConfigManager.configPath) as an array of ConfigModelProvider objects. This schema includes the provider id, type, chatModels, embeddingModels, and encrypted config values, enabling the ModelRegistry to rebuild active provider instances across server restarts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}