# Understanding the Base Interface for LLM and Embedding Models in Vane

> Discover the Base Interface for LLM and Embedding Models in Vane. Learn about abstract base classes and provider implementation requirements.

- Repository: [Kushagra Srivastava/Vane](https://github.com/ItzCrazyKns/Vane)
- Tags: internals
- Published: 2026-03-11

---

**Vane defines abstract base classes `BaseLLM<CONFIG>` and `BaseEmbedding<CONFIG>` in `src/lib/models/base/` that serve as the contract all language model and embedding providers must implement.**

The [ItzCrazyKns/Vane](https://github.com/ItzCrazyKns/Vane) repository uses a provider-pattern architecture to support multiple LLM and embedding backends. At the heart of this system are two generic abstract classes that standardize how the application interacts with models, enabling seamless swapping between OpenAI, Ollama, Anthropic, and custom providers without changing consumer code.

## Core Abstractions: BaseLLM and BaseEmbedding

Vane’s model layer is built around two primary interfaces located in `src/lib/models/base/`. Both use TypeScript generics to allow type-safe configuration while enforcing a uniform public API.

### The LLM Contract: BaseLLM

The `BaseLLM<CONFIG>` abstract class in [[`src/lib/models/base/llm.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/base/llm.ts)](https://github.com/ItzCrazyKns/Vane/blob/master/src/lib/models/base/llm.ts) defines the interface for all text generation capabilities. Concrete providers must implement four abstract methods:

```typescript
abstract class BaseLLM<CONFIG> {
  constructor(protected config: CONFIG) {}

  /** Generate a single text completion */
  abstract generateText(input: GenerateTextInput): Promise<GenerateTextOutput>;

  /** Stream a text completion token‑by‑token */
  abstract streamText(
    input: GenerateTextInput,
  ): AsyncGenerator<StreamTextOutput>;

  /** Generate a structured object using a Zod schema */
  abstract generateObject<T>(input: GenerateObjectInput): Promise<z.infer<T>>;

  /** Stream a structured object (partial results) */
  abstract streamObject<T>(
    input: GenerateObjectInput,
  ): AsyncGenerator<Partial<z.infer<T>>>;
}

```

All concrete LLM providers—whether wrapping OpenAI’s GPT-4, Ollama’s local models, or Anthropic’s Claude—extend `BaseLLM` and provide implementations for these four operations. The generic `CONFIG` parameter allows each provider to maintain its own strongly-typed settings (API keys, model identifiers, temperature values) while exposing the same consumer interface.

### The Embedding Contract: BaseEmbedding

The `BaseEmbedding<CONFIG>` abstract class in [[`src/lib/models/base/embedding.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/base/embedding.ts)](https://github.com/ItzCrazyKns/Vane/blob/master/src/lib/models/base/embedding.ts) standardizes vectorization of text. It declares two abstract methods for converting content into numerical embeddings:

```typescript
abstract class BaseEmbedding<CONFIG> {
  constructor(protected config: CONFIG) {}

  /** Embed a list of raw strings */
  abstract embedText(texts: string[]): Promise<number[][]>;

  /** Embed an array of `Chunk` objects (text + metadata) */
  abstract embedChunks(chunks: Chunk[]): Promise<number[][]>;
}

```

Embedding providers for OpenAI, Ollama, HuggingFace, and other backends implement these methods to ensure that Vane’s retrieval-augmented generation (RAG) pipeline receives consistent vector representations regardless of the underlying model.

## Implementing Custom Model Providers

The base interfaces enable developers to add support for new model backends without modifying core application logic.

### Creating a Custom LLM Provider

To integrate a proprietary or niche LLM, extend `BaseLLM` and implement the four required methods:

```typescript
import BaseLLM from '@/lib/models/base/llm';
import {
  GenerateTextInput,
  GenerateTextOutput,
  GenerateObjectInput,
  StreamTextOutput,
} from '@/lib/types';
import { z } from 'zod';

interface MyLLMConfig {
  apiKey: string;
  endpoint: string;
}

class MyLLM extends BaseLLM<MyLLMConfig> {
  async generateText(input: GenerateTextInput): Promise<GenerateTextOutput> {
    // Implementation calling custom API
    return { text: 'response', usage: { tokens: 10 } };
  }

  async *streamText(
    input: GenerateTextInput,
  ): AsyncGenerator<StreamTextOutput> {
    // Yield tokens as they arrive from the API
    yield { token: 'Hello' };
    yield { token: ' world' };
  }

  async generateObject<T>(input: GenerateObjectInput): Promise<z.infer<T>> {
    const schema = input.schema as unknown as z.ZodType<any>;
    const raw = await this.generateText(input);
    return schema.parse(JSON.parse(raw.text));
  }

  async *streamObject<T>(
    input: GenerateObjectInput,
  ): AsyncGenerator<Partial<z.infer<T>>> {
    // Stream partial JSON objects
    for await (const chunk of this.streamText(input)) {
      yield {}; // Partial parsing logic here
    }
  }
}

```

### Creating a Custom Embedding Provider

Similarly, custom embedding backends extend `BaseEmbedding`:

```typescript
import BaseEmbedding from '@/lib/models/base/embedding';
import type { Chunk } from '@/lib/types';

interface MyEmbeddingConfig {
  modelId: string;
}

class MyEmbedding extends BaseEmbedding<MyEmbeddingConfig> {
  async embedText(texts: string[]): Promise<number[][]> {
    return texts.map((t) => this.vectorize(t));
  }

  async embedChunks(chunks: Chunk[]): Promise<number[][]> {
    return chunks.map((c) => this.vectorize(c.text));
  }

  private vectorize(text: string): number[] {
    // Naive example: character code vectors
    return Array.from(text).map((ch) => ch.charCodeAt(0) % 10);
  }
}

```

## Provider Architecture and File Organization

Vane’s model layer is organized to separate contracts from implementations:

| File | Role |
|------|------|
| [[`src/lib/models/base/llm.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/base/llm.ts)](https://github.com/ItzCrazyKns/Vane/blob/master/src/lib/models/base/llm.ts) | Abstract LLM contract (`BaseLLM`). |
| [[`src/lib/models/base/embedding.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/base/embedding.ts)](https://github.com/ItzCrazyKns/Vane/blob/master/src/lib/models/base/embedding.ts) | Abstract embedding contract (`BaseEmbedding`). |
| [[`src/lib/models/base/provider.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/base/provider.ts)](https://github.com/ItzCrazyKns/Vane/blob/master/src/lib/models/base/provider.ts) | Common provider base (`BaseModelProvider`) that coordinates LLM and embedding loading. |
| `src/lib/models/providers/*` | Concrete implementations (OpenAI, Ollama, Anthropic, etc.) that extend the base classes. |

This structure ensures that adding a new model provider requires only implementing the abstract methods in `BaseLLM` or `BaseEmbedding`, without touching the core retrieval or generation logic elsewhere in the application.

## Summary

- **BaseLLM<CONFIG>** in [`src/lib/models/base/llm.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/base/llm.ts) defines the contract for text generation, streaming, and structured object generation.
- **BaseEmbedding<CONFIG>** in [`src/lib/models/base/embedding.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/base/embedding.ts) defines the contract for vectorizing text and document chunks.
- Both use TypeScript generics to allow type-safe, provider-specific configurations while maintaining a uniform public API.
- Concrete providers in `src/lib/models/providers/*` extend these abstract classes to integrate OpenAI, Ollama, Anthropic, and other backends.

## Frequently Asked Questions

### What is the purpose of the generic CONFIG parameter in BaseLLM and BaseEmbedding?

The `CONFIG` generic allows each provider to define its own strongly-typed configuration object—such as API keys, model identifiers, or endpoint URLs—while the abstract base class enforces a consistent constructor signature and public API. This enables type-safe access to provider-specific settings within concrete implementations without breaking the uniform interface that the rest of Vane expects.

### How does Vane handle both text generation and structured object generation?

The `BaseLLM` interface declares separate methods for these use cases: `generateText` and `streamText` handle raw string completions, while `generateObject` and `streamObject` accept a Zod schema and return validated, typed objects. This separation allows providers to optimize API calls—using JSON mode or function calling where available—while presenting a consistent interface to Vane’s agent and RAG pipelines.

### What is the difference between embedText and embedChunks in the embedding interface?

`embedText` accepts an array of raw strings and returns their vector representations, suitable for simple queries or standalone documents. `embedChunks` accepts an array of `Chunk` objects—which include both text and metadata—and is optimized for embedding structured document segments during indexing. Both methods return `Promise<number[][]>` to ensure consistent downstream consumption by Vane’s vector stores.

### How do BaseLLM and BaseEmbedding relate to BaseModelProvider?

While `BaseLLM` and `BaseEmbedding` define the operational contracts for generation and vectorization, `BaseModelProvider` in [`src/lib/models/base/provider.ts`](https://github.com/ItzCrazyKns/Vane/blob/main/src/lib/models/base/provider.ts) serves as a higher-level coordinator that typically manages the lifecycle, configuration loading, and instantiation of both LLM and embedding instances. Concrete providers usually extend or compose these three base classes to offer a complete, unified backend service to the Vane application.