How OmniRoute's Vector-Based Memory System Stores and Retrieves Context

OmniRoute stores conversational memories in SQLite and mirrors each entry as a vector embedding in a sqlite-vec virtual table, enabling fast K-nearest-neighbor and hybrid searches with graceful degradation to full-text search when native extensions are unavailable.

The vector-based memory system in the open-source diegosouzapw/OmniRoute repository persists conversational context in a local SQLite database. By combining sqlite-vec virtual tables with traditional FTS5 indexes, it supports both semantic similarity queries and keyword-based retrieval without requiring an external vector database.

Vector-Based Memory Storage Pipeline

Resolving the Embedding Source

When a memory is created or updated, scheduleVectorUpsert() in src/lib/memory/store.ts resolves the active embedding configuration via resolveEmbeddingSource(settings), implemented in src/lib/memory/embedding/index.ts, and calls embed() to obtain a Float32Array vector.

Upserting Vectors into sqlite-vec

The generated vector is handed to the singleton VectorStore, obtained via getVectorStore() in src/lib/memory/vectorStore.ts. Before any write, ensureReady() checks whether the vec_memories virtual table exists and whether its schema matches the current embedding signature. If the dimensionality or quantization mode has changed, the table is recreated on-the-fly.

upsertVector() maps the memory's UUID to the SQLite rowid and inserts or replaces the embedding using a DELETE + INSERT pattern required by the vec0 module.

Synchronizing Deletions

When a memory is removed, deleteVector() drops the corresponding row from vec_memories. This keeps the vector index synchronized with the primary memories table and prevents orphaned embeddings from appearing in search results.

Context Retrieval Methods

Pure Vector KNN Search

For semantic lookups, searchVector() in src/lib/memory/vectorStore.ts executes a K-nearest-neighbor query using v.embedding MATCH ? against vec_memories. Hits are ordered by L2 distance and returned as objects containing memoryId, distance, and score.

Hybrid Vector and Full-Text Search

searchHybrid() runs two parallel queries: a vector KNN against vec_memories and a full-text search against the memory_fts FTS5 table. It then merges the two result sets using Reciprocal Rank Fusion (RRF) to produce a single ranked list that balances semantic similarity with keyword relevance.

Fallback Without Native Extensions

If the sqlite-vec extension cannot be loaded—for example, in WASM or restricted cloud environments—getVectorStore() returns null. The request pipeline, coordinated through helpers in src/lib/memory/retrieval.ts, degrades to pure FTS5 keyword search against memory_fts, ensuring the memory system remains functional even without vector support.

Managing Schema Changes and Re-indexing

When the embedding signature changes—such as switching models or toggling int8 quantization—resetForSignature() drops and recreates the vec_memories virtual table. It marks every existing memory for re-indexing by updating memory_vec_meta via getMemoryVecMeta() and setMemoryVecMeta(), re-exported from src/lib/localDb.ts. The next access to each memory triggers a fresh vector generation through scheduleVectorUpsert().

Practical Code Examples

Creating a Memory

Vectors are generated asynchronously in the background after the primary row is inserted.

import { createMemory } from "@/lib/memory/store";

await createMemory({
  apiKeyId: "key-123",
  sessionId: "sess-abc",
  type: "user",
  key: "topic-1",
  content: "Explain how vector stores work in OmniRoute",
  metadata: { source: "chat" },
});

Running a Pure Vector Search

import { getVectorStore } from "@/lib/memory/vectorStore";
import { resolveEmbeddingSource } from "@/lib/memory/embedding";

const settings = await getMemorySettings();
const resolution = resolveEmbeddingSource(settings);
const vec = getVectorStore();

if (vec) {
  await vec.ensureReady(resolution);
  const hits = await vec.searchVector(resolution.exampleVector, 10);
  console.log(hits); // [{ memoryId, distance, score }, …]
}

Running a Hybrid Search

import { getVectorStore } from "@/lib/memory/vectorStore";

const vec = getVectorStore();
if (vec) {
  const hits = await vec.searchHybrid(
    resolution.exampleVector,  // query embedding
    "vector store",            // keyword query
    10,                        // top-K
    "key-123",                 // optional API-key filter
  );
  console.log(hits);
}

Deleting a Memory and Its Vector

import { deleteMemory } from "@/lib/memory/store";

await deleteMemory("c3f4a7b2-e1d9-4f2a-a9c1-7f6e5d8b9a0c");

Summary

• OmniRoute's vector-based memory system uses SQLite for primary storage and a sqlite-vec virtual table named vec_memories for embeddings.
• scheduleVectorUpsert() in src/lib/memory/store.ts triggers background embedding via resolveEmbeddingSource() and embed().
• VectorStore.upsertVector() in src/lib/memory/vectorStore.ts writes vectors using a DELETE + INSERT pattern required by vec0.
• searchVector() performs KNN lookup by L2 distance, while searchHybrid() fuses KNN and FTS5 results with Reciprocal Rank Fusion.
• If sqlite-vec fails to load, the system falls back to pure FTS5 keyword search through src/lib/memory/retrieval.ts.
• Schema changes automatically invalidate vectors through resetForSignature() and memory_vec_meta tracking.

Frequently Asked Questions

How does OmniRoute generate embeddings for memories?

When a memory is created or updated, scheduleVectorUpsert() in src/lib/memory/store.ts resolves the active embedding source via resolveEmbeddingSource(), defined in src/lib/memory/embedding/index.ts, and calls embed() to produce a Float32Array vector. This process runs asynchronously so the primary memory row remains available immediately.

What happens if the sqlite-vec extension is not available?

If the native extension cannot be loaded, getVectorStore() returns null and the request pipeline degrades to pure FTS5 keyword search on the memory_fts table. This ensures memory retrieval remains functional in WASM or restricted cloud environments.

How does OmniRoute handle changes to the embedding model?

resetForSignature() detects signature changes—such as a new model or quantization setting—and drops and recreates the vec_memories table. It updates memory_vec_meta via helpers in src/lib/localDb.ts to mark all memories for re-indexing, and vectors are regenerated on the next access.

What is the difference between pure vector and hybrid search?

Pure vector search uses searchVector() to query vec_memories with v.embedding MATCH ? and ranks results by L2 distance. Hybrid search uses searchHybrid() to execute both a vector KNN and an FTS5 keyword query, then merges the result sets with Reciprocal Rank Fusion to balance semantic similarity and keyword relevance.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}