How Vane Uses Embedding Models for Semantic Search: Architecture and Implementation

Vane implements semantic search by converting both uploaded documents and user queries into dense vector embeddings, then ranking results using cosine similarity through a modular, provider-agnostic pipeline.

Vane enables semantic search over user-uploaded files by leveraging interchangeable embedding models to vectorize text content. According to the ItzCrazyKns/Vane source code, the system embeds document chunks during ingestion, stores vectors in JSON side-car files, and performs in-memory similarity scoring to surface contextually relevant information.

The Semantic Search Pipeline

Vane’s semantic search capability is built on three tightly-coupled components that handle ingestion, storage, and retrieval.

File Ingestion and Chunk Embedding

When users upload files, the UploadManager class processes them through the processFiles method. In src/lib/uploads/manager.ts (lines 94-106, 122-136, 150-164), the extractContentAndEmbed workflow splits documents into text chunks and invokes this.embeddingModel.embedText(...) to generate dense vector representations for each chunk.

Vector Storage and Indexing

The system persists embeddings alongside their text content in *.content.json side-car files. During initialization, UploadStore.initializeStore (lines 31-45 in src/lib/uploads/store.ts) loads these JSON files into memory, building an index where each record contains both the original chunk content and its corresponding embedding vector.

Query Processing and Similarity Scoring

For retrieval operations, Vane embeds the user query using the same model via UploadStore.query (lines 54-57). The system then calculates semantic relevance using computeSimilarity(query, record.embedding) (lines 60-71), which implements cosine similarity to compare the query vector against stored chunk embeddings. Scores are normalized, aggregated across multiple queries when necessary, and sorted to return the top-K most relevant chunks (lines 78-96).

Provider-Agnostic Embedding Architecture

Vane abstracts embedding providers through the BaseEmbedding class defined in src/lib/models/base/embedding.ts (lines 3-6). Concrete implementations such as OpenAIEmbedding in src/lib/models/providers/openai/openaiEmbedding.ts (lines 23-31) expose standardized embedText and embedChunks methods. This abstraction allows the semantic search pipeline to interchange OpenAI, Ollama, LM-Studio, or custom providers without modifying core search logic.

Implementation Examples

Creating an Embedding Model

import OpenAIEmbedding from '@/lib/models/providers/openai/openaiEmbedding';

const embedding = new OpenAIEmbedding({
  apiKey: process.env.OPENAI_API_KEY!,
  model: 'text-embedding-3-small',
});

Ingesting Files and Storing Embeddings

const manager = new UploadManager({ embeddingModel: embedding });
await manager.processFiles([pdfFile]);   // splits, embeds, writes *.content.json

Performing Semantic Search

const store = new UploadStore({
  embeddingModel: embedding,
  fileIds: ['<uploaded-file-id>'],
});

const results = await store.query(
  ['What are the key challenges described in the report?'],
  5,                     // return top-5 chunks
);
console.log(results);   // array of { content, metadata }

Adding a Custom Provider

class MyEmbedding extends BaseEmbedding<MyConfig> {
  async embedText(texts: string[]) {
    // call your own vector service here
    return await myVectorService.encode(texts);
  }
  async embedChunks(chunks: Chunk[]) {
    return this.embedText(chunks.map(c => c.content));
  }
}

Register this class in a provider module following the pattern in src/lib/models/providers/openai/index.ts to integrate custom embedding models for semantic search.

Summary

• Vane embeds both document chunks and user queries using the abstract BaseEmbedding interface, enabling provider interchangeability.
• Vector representations are stored in JSON side-car files (*.content.json) and loaded into UploadStore for efficient in-memory retrieval.
• Cosine similarity calculations in computeSimilarity rank chunks by semantic relevance to the embedded query.
• The pipeline supports multiple embedding models including OpenAI's text-embedding-3-small and text-embedding-3-large, configurable through the provider registration system.

Frequently Asked Questions

What embedding models does Vane support by default?

Vane registers OpenAI's text-embedding-3-small and text-embedding-3-large models through the provider system in src/lib/models/providers/openai/index.ts (lines 98-106). The architecture supports any provider implementing the BaseEmbedding interface, allowing integration of Ollama, LM-Studio, or custom services.

How does Vane calculate semantic similarity between queries and documents?

Vane uses cosine similarity to compare the query embedding against stored chunk embeddings. The computeSimilarity function—called within UploadStore.query at lines 60-71 of src/lib/uploads/store.ts—calculates vector similarity scores, which are then normalized and ranked to retrieve the most semantically relevant text chunks.

Can I use a local or custom embedding provider instead of OpenAI?

Yes. Extend the abstract BaseEmbedding class from src/lib/models/base/embedding.ts and implement the embedText and embedChunks methods to interface with your preferred vector service. Register the provider following the pattern in the OpenAI provider module to make your custom embedding models available for semantic search throughout Vane.

Where does Vane store the generated embeddings?

Vane writes embeddings to *.content.json side-car files alongside uploaded documents. The UploadStore class reads these files during initialization (lines 31-45 of src/lib/uploads/store.ts) to build an in-memory index, enabling fast similarity calculations without repeated API calls to embedding providers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}