Understanding the Base Interface for LLM and Embedding Models in Vane

Vane defines abstract base classes BaseLLM<CONFIG> and BaseEmbedding<CONFIG> in src/lib/models/base/ that serve as the contract all language model and embedding providers must implement.

The ItzCrazyKns/Vane repository uses a provider-pattern architecture to support multiple LLM and embedding backends. At the heart of this system are two generic abstract classes that standardize how the application interacts with models, enabling seamless swapping between OpenAI, Ollama, Anthropic, and custom providers without changing consumer code.

Core Abstractions: BaseLLM and BaseEmbedding

Vane’s model layer is built around two primary interfaces located in src/lib/models/base/. Both use TypeScript generics to allow type-safe configuration while enforcing a uniform public API.

The LLM Contract: BaseLLM

The BaseLLM<CONFIG> abstract class in [src/lib/models/base/llm.ts](https://github.com/ItzCrazyKns/Vane/blob/master/src/lib/models/base/llm.ts) defines the interface for all text generation capabilities. Concrete providers must implement four abstract methods:

abstract class BaseLLM<CONFIG> {
  constructor(protected config: CONFIG) {}

  /** Generate a single text completion */
  abstract generateText(input: GenerateTextInput): Promise<GenerateTextOutput>;

  /** Stream a text completion token‑by‑token */
  abstract streamText(
    input: GenerateTextInput,
  ): AsyncGenerator<StreamTextOutput>;

  /** Generate a structured object using a Zod schema */
  abstract generateObject<T>(input: GenerateObjectInput): Promise<z.infer<T>>;

  /** Stream a structured object (partial results) */
  abstract streamObject<T>(
    input: GenerateObjectInput,
  ): AsyncGenerator<Partial<z.infer<T>>>;
}

All concrete LLM providers—whether wrapping OpenAI’s GPT-4, Ollama’s local models, or Anthropic’s Claude—extend BaseLLM and provide implementations for these four operations. The generic CONFIG parameter allows each provider to maintain its own strongly-typed settings (API keys, model identifiers, temperature values) while exposing the same consumer interface.

The Embedding Contract: BaseEmbedding

The BaseEmbedding<CONFIG> abstract class in [src/lib/models/base/embedding.ts](https://github.com/ItzCrazyKns/Vane/blob/master/src/lib/models/base/embedding.ts) standardizes vectorization of text. It declares two abstract methods for converting content into numerical embeddings:

abstract class BaseEmbedding<CONFIG> {
  constructor(protected config: CONFIG) {}

  /** Embed a list of raw strings */
  abstract embedText(texts: string[]): Promise<number[][]>;

  /** Embed an array of `Chunk` objects (text + metadata) */
  abstract embedChunks(chunks: Chunk[]): Promise<number[][]>;
}

Embedding providers for OpenAI, Ollama, HuggingFace, and other backends implement these methods to ensure that Vane’s retrieval-augmented generation (RAG) pipeline receives consistent vector representations regardless of the underlying model.

Implementing Custom Model Providers

The base interfaces enable developers to add support for new model backends without modifying core application logic.

Creating a Custom LLM Provider

To integrate a proprietary or niche LLM, extend BaseLLM and implement the four required methods:

import BaseLLM from '@/lib/models/base/llm';
import {
  GenerateTextInput,
  GenerateTextOutput,
  GenerateObjectInput,
  StreamTextOutput,
} from '@/lib/types';
import { z } from 'zod';

interface MyLLMConfig {
  apiKey: string;
  endpoint: string;
}

class MyLLM extends BaseLLM<MyLLMConfig> {
  async generateText(input: GenerateTextInput): Promise<GenerateTextOutput> {
    // Implementation calling custom API
    return { text: 'response', usage: { tokens: 10 } };
  }

  async *streamText(
    input: GenerateTextInput,
  ): AsyncGenerator<StreamTextOutput> {
    // Yield tokens as they arrive from the API
    yield { token: 'Hello' };
    yield { token: ' world' };
  }

  async generateObject<T>(input: GenerateObjectInput): Promise<z.infer<T>> {
    const schema = input.schema as unknown as z.ZodType<any>;
    const raw = await this.generateText(input);
    return schema.parse(JSON.parse(raw.text));
  }

  async *streamObject<T>(
    input: GenerateObjectInput,
  ): AsyncGenerator<Partial<z.infer<T>>> {
    // Stream partial JSON objects
    for await (const chunk of this.streamText(input)) {
      yield {}; // Partial parsing logic here
    }
  }
}

Creating a Custom Embedding Provider

Similarly, custom embedding backends extend BaseEmbedding:

import BaseEmbedding from '@/lib/models/base/embedding';
import type { Chunk } from '@/lib/types';

interface MyEmbeddingConfig {
  modelId: string;
}

class MyEmbedding extends BaseEmbedding<MyEmbeddingConfig> {
  async embedText(texts: string[]): Promise<number[][]> {
    return texts.map((t) => this.vectorize(t));
  }

  async embedChunks(chunks: Chunk[]): Promise<number[][]> {
    return chunks.map((c) => this.vectorize(c.text));
  }

  private vectorize(text: string): number[] {
    // Naive example: character code vectors
    return Array.from(text).map((ch) => ch.charCodeAt(0) % 10);
  }
}

Provider Architecture and File Organization

Vane’s model layer is organized to separate contracts from implementations:

File	Role
[src/lib/models/base/llm.ts](https://github.com/ItzCrazyKns/Vane/blob/master/src/lib/models/base/llm.ts)	Abstract LLM contract (BaseLLM).
[src/lib/models/base/embedding.ts](https://github.com/ItzCrazyKns/Vane/blob/master/src/lib/models/base/embedding.ts)	Abstract embedding contract (BaseEmbedding).
[src/lib/models/base/provider.ts](https://github.com/ItzCrazyKns/Vane/blob/master/src/lib/models/base/provider.ts)	Common provider base (BaseModelProvider) that coordinates LLM and embedding loading.
src/lib/models/providers/*	Concrete implementations (OpenAI, Ollama, Anthropic, etc.) that extend the base classes.

This structure ensures that adding a new model provider requires only implementing the abstract methods in BaseLLM or BaseEmbedding, without touching the core retrieval or generation logic elsewhere in the application.

Summary

• BaseLLM in src/lib/models/base/llm.ts defines the contract for text generation, streaming, and structured object generation.
• BaseEmbedding in src/lib/models/base/embedding.ts defines the contract for vectorizing text and document chunks.
• Both use TypeScript generics to allow type-safe, provider-specific configurations while maintaining a uniform public API.
• Concrete providers in src/lib/models/providers/* extend these abstract classes to integrate OpenAI, Ollama, Anthropic, and other backends.

Frequently Asked Questions

What is the purpose of the generic CONFIG parameter in BaseLLM and BaseEmbedding?

The CONFIG generic allows each provider to define its own strongly-typed configuration object—such as API keys, model identifiers, or endpoint URLs—while the abstract base class enforces a consistent constructor signature and public API. This enables type-safe access to provider-specific settings within concrete implementations without breaking the uniform interface that the rest of Vane expects.

How does Vane handle both text generation and structured object generation?

The BaseLLM interface declares separate methods for these use cases: generateText and streamText handle raw string completions, while generateObject and streamObject accept a Zod schema and return validated, typed objects. This separation allows providers to optimize API calls—using JSON mode or function calling where available—while presenting a consistent interface to Vane’s agent and RAG pipelines.

What is the difference between embedText and embedChunks in the embedding interface?

embedText accepts an array of raw strings and returns their vector representations, suitable for simple queries or standalone documents. embedChunks accepts an array of Chunk objects—which include both text and metadata—and is optimized for embedding structured document segments during indexing. Both methods return Promise<number[][]> to ensure consistent downstream consumption by Vane’s vector stores.

How do BaseLLM and BaseEmbedding relate to BaseModelProvider?

While BaseLLM and BaseEmbedding define the operational contracts for generation and vectorization, BaseModelProvider in src/lib/models/base/provider.ts serves as a higher-level coordinator that typically manages the lifecycle, configuration loading, and instantiation of both LLM and embedding instances. Concrete providers usually extend or compose these three base classes to offer a complete, unified backend service to the Vane application.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}