How LLM Wiki Implements Vector Semantic Search with LanceDB: A Technical Deep Dive

LLM Wiki stores and retrieves embeddings in a local LanceDB vector store within its Tauri Rust backend, using delete-then-insert upsert semantics and cosine similarity vectors to return semantically relevant wiki chunks.

The nashsu/llm_wiki repository demonstrates a production-ready implementation of on-device vector semantic search using LanceDB as its embedded vector database. This architecture enables fast, local retrieval of wiki content based on meaning rather than keyword matching, all while maintaining data consistency through careful upsert operations in Rust.

Vector Storage Architecture

LanceDB Table Schema

The vector store persists embeddings in a local LanceDB table located at ~/.llm-wiki/lancedb. Each row in the table follows a strict schema designed to support semantic retrieval while maintaining referential integrity to source wiki pages:

• page_id (string): Unique identifier linking the embedding back to its source wiki page
• vector (float-32 array): Dense embedding generated by external LLM endpoints (OpenAI, Gemini, etc.)
• chunk_text (string): Original text content that was embedded, preserved for result snippets
• Metadata: Optional fields including headings and timestamps for additional filtering capabilities

Dependency Configuration

The Rust backend declares LanceDB as a core dependency in src-tauri/Cargo.toml with version pinning for stability:

[dependencies]
lancedb = "0.27.2"

The codebase imports specific LanceDB traits to enable advanced query capabilities: lancedb::connect, lancedb::query::{ExecutableQuery, QueryBase}, and lancedb::table::{CompactionOptions, OptimizeAction}.

Indexing Workflow and Upsert Semantics

The Delete-Then-Insert Pattern

In src-tauri/src/commands/vectorstore.rs, LLM Wiki implements custom upsert semantics to ensure data consistency when updating page embeddings. Rather than blindly appending new vectors, the code first purges all existing rows associated with a page before inserting fresh embeddings.

As documented in the comment on line 368: "Upsert semantics: we DELETE every row with the target page_id and then" insert the new records. This guarantees that outdated embeddings never persist alongside current content, preventing stale search results.

The implementation uses LanceDB's filter-based deletion API:

// Remove existing embeddings for this page to prevent duplicates
table.delete()
    .filter("page_id = ?", &page_id)
    .execute()?;

// Prepare new records from embedding vectors
let records = embeddings.iter().map(|e| {
    Record {
        page_id: page_id.clone(),
        vector: e.vector.clone(),
        chunk_text: e.text.clone(),
        // additional metadata...
    }
}).collect::<Vec<_>>();

// Insert fresh embeddings
table.add(&records).execute()?;

This pattern ensures atomic replacement of a page's vector representation without requiring complex merge logic or transaction management.

Querying with Vector Semantic Search

Cosine Similarity Search Implementation

The src-tauri/src/commands/search.rs file implements the retrieval side of the vector semantic search pipeline. When a user submits a query, the frontend invokes the Rust search command, which embeds the query string using the same model configuration and executes a similarity search against the LanceDB table.

The query construction leverages lancedb::Metric::Cosine for measuring vector similarity, providing results ranked by semantic proximity rather than lexical overlap:

// Generate embedding for the user query
let query_vec = embed(query_string)?;

// Execute vector similarity search
let results = table
    .search()
    .vector(&query_vec)
    .metric(lancedb::Metric::Cosine)  // Cosine similarity metric
    .limit(k)                          // Return top-k matches
    .execute()?;

// Process results containing page metadata and similarity scores
for hit in results {
    println!("Page: {}, Score: {}, Snippet: {}",
             hit.page_id, hit.score, hit.chunk_text);
}

The ExecutableQuery trait enables this fluent API, allowing method chaining for vector specification, metric selection, and result limiting before execution.

Testing and Validation

The repository includes comprehensive validation in src/lib/embedding.real-llm.test.ts to ensure the vector semantic search returns meaningfully relevant results. Integration tests verify that the embedding generation pipeline produces vectors that correctly capture semantic relationships between queries and stored chunks.

Lines 1990-2012 in the test suite demonstrate this validation by asserting that a query returns a specific chunk containing the phrase "semantic chunk". The test confirms that the LanceDB retrieval pipeline returns chunk_text values that semantically align with the query intent, validating the end-to-end vector search functionality.

Summary

• Local-First Architecture: LLM Wiki uses LanceDB 0.27.2 as an embedded vector store within the Tauri Rust backend, eliminating network latency for search operations.
• Consistent Upserts: The vectorstore.rs command implements delete-then-insert semantics (line 368) to ensure atomic updates of page embeddings without stale data.
• Cosine Similarity: The search.rs command executes vector queries using Metric::Cosine to rank results by semantic meaning rather than keyword density.
• Schema Design: The table stores page_id, vector, and chunk_text to enable both similarity computation and human-readable result presentation.
• Test Coverage: Integration tests in embedding.real-llm.test.ts (lines 1990-2012) validate that the semantic search returns contextually relevant wiki chunks.

Frequently Asked Questions

How does LLM Wiki handle updating embeddings when page content changes?

When a page is re-indexed, the vectorstore.rs command first deletes all existing rows matching the page_id using table.delete().filter("page_id = ?", ...), then inserts the new embedding vectors. This delete-then-insert pattern ensures the vector store never contains stale embeddings from previous versions of the page.

What similarity metric does LLM Wiki use for semantic search?

According to the implementation in src-tauri/src/commands/search.rs, LLM Wiki uses cosine similarity via lancedb::Metric::Cosine to measure vector proximity. Cosine similarity is effective for semantic search because it focuses on the orientation of vectors in high-dimensional space rather than their magnitude, capturing semantic meaning regardless of text length.

Where is the LanceDB database file stored on disk?

The vector store persists data in a local directory at ~/.llm-wiki/lancedb within the user's home folder. This location is accessed through LanceDB's connect API in the Rust backend, creating a self-contained, on-device search index that requires no external database server.

What embedding models does LLM Wiki support?

While the vector storage layer in vectorstore.rs and search.rs is model-agnostic, the frontend integration tests indicate support for external LLM endpoints including OpenAI and Gemini. The system generates embeddings through these APIs before passing the dense vectors to the Rust backend for storage in LanceDB.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}