How OmniRoute's Memory System Uses FTS5 and Vector Quantization: A Technical Deep Dive

OmniRoute implements a three-tier retrieval hierarchy that combines SQLite FTS5 full-text search with optional int8 vector quantization via sqlite-vec or Qdrant, automatically degrading to keyword search when vector stores are unavailable.

OmniRoute stores conversational memories in a SQLite database and retrieves the most relevant ones using a hybrid architecture that balances precision, performance, and fault tolerance. The system is designed according to the source code in diegosouzapw/OmniRoute to prioritize exact keyword matches as a universal fallback while leveraging quantized vector embeddings for semantic similarity when available.

Three-Tier Retrieval Architecture

The memory engine selects backends dynamically based on availability and configuration, forming a degradation chain from fastest to most resilient.

Tier 0: FTS5 Keyword Search (Always Available)

The foundation of OmniRoute's memory system is a SQLite FTS5 virtual table that provides immediate text-based retrieval without external dependencies. During database migration 022_add_memory_fts5.sql, the system creates a virtual table named memory_fts indexing the content and key columns of the memories table.

A subsequent migration, 023_fix_memory_fts_uuid.sql, resolves the impedance mismatch between SQLite's integer rowid requirement and OmniRoute's UUID primary keys. It adds a memory_id column to the FTS table and establishes triggers that synchronize inserts, updates, and deletes between the main memories table and the FTS index.

In src/lib/memory/retrieval.ts, the retrieval logic constructs queries using memory_fts MATCH ? syntax. For exact search strategies, or when vector stores fail to load, the engine executes a ranked FTS5 query ordered by the built-in rank function. If the FTS table returns no matches or is missing, the system falls back to a chronological scan using ORDER BY created_at.

The engineStatus() API explicitly reports keyword.backend = "FTS5" and keyword.available = true, confirming this tier is permanently active.

Tier 1: sqlite-vec with Float32 and int8 Quantization

When the sqlite-vec extension loads successfully and an embedding source is configured (remote API, static Potion model, or local Transformers.js), the system activates vector search. The vectorStore.ts module creates a table vec_memories and populates it with embeddings.

Vector quantization is controlled via the MEMORY_VEC_QUANTIZATION environment variable. When set to "int8", the insertion code in vectorStore.ts calls vec_quantize_int8(?, 'unit'), compressing vectors to 8-bit integers. This reduces storage footprint by approximately 4× compared to Float32, with a modest trade-off in recall accuracy.

Retrieval executes brute-force K-NN search using SELECT * FROM vec_memories ORDER BY distance, returning semantically similar memories based on cosine distance.

Tier 2: Qdrant External Vector Database

If enabled in the Engine settings (qdrantEnabled), OmniRoute elevates Qdrant to the primary vector backend. The qdrant.ts module configures scalar quantization through the normalizeQdrantConfig function, reading the qdrantQuantization setting ("none", "int8", or "binary").

For int8 mode, the driver constructs a scalar quantization configuration with always_ram: true and quantile 0.99, enabling rescore: true to refine results using full-precision vectors during final ranking. This ensures that while storage is compressed, retrieval accuracy remains high through re-scoring.

Hybrid Ranking with Reciprocal Rank Fusion

When both FTS5 and vector stores are available, OmniRoute employs Reciprocal Rank Fusion (RRF) to combine results. The implementation in retrieval.ts uses a default constant of k = 60, computing composite scores as:

score = Σ (1 / (k + rank_i))

Where rank_i is the position of the document in each individual ranking list (FTS5 and vector). This produces a unified relevance score that balances keyword density with semantic similarity, eliminating the need to manually tune weighting parameters.

Lazy Re-indexing and Quantization Migration

Changing quantization modes does not block startup. When MEMORY_VEC_QUANTIZATION changes, the system marks existing vectors with needs_reindex = 1 in the memory_vec_meta table. The next retrieval operation triggers a lazy backfill that recomputes embeddings in the new format, allowing runtime configuration changes without downtime.

Configuration and Code Examples

Enable FTS5-Based Memory Retrieval

FTS5 is active by default once migrations run. Verify status via the API:

curl -X GET http://localhost:20128/api/memory/engine-status \
  -H "Authorization: Bearer $OMNIROUTE_KEY"

Expected response includes:

{
  "keyword": {
    "backend": "FTS5",
    "available": true
  }
}

Enable int8 Quantization with sqlite-vec

Set the environment variable before starting the server:

export MEMORY_VEC_QUANTIZATION=int8
npm start

This forces a re-index of vec_memories; the next retrieval lazily recomputes embeddings in int8 format.

Configure Qdrant with int8 Quantization

curl -X PUT http://localhost:20128/api/settings/qdrant \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -d '{
    "enabled": true,
    "host": "qdrant.local",
    "port": 6333,
    "collection": "omniroute_memory",
    "quantization": "int8"
  }'

Execute Hybrid Retrieval

curl -X POST http://localhost:20128/api/memory/retrieve-preview \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -d '{
    "strategy": "hybrid",
    "query": "TypeScript logging best practices",
    "maxTokens": 1500
  }'

The response includes score, tier (indicating FTS5 or vector origin), and the active quantization mode.

Summary

• FTS5 is the invariant backbone: Implemented via 022_add_memory_fts5.sql and 023_fix_memory_fts_uuid.sql, it provides keyword search with UUID-to-rowid mapping and automatic synchronization triggers.
• Vector quantization reduces footprint: Both sqlite-vec (vec_quantize_int8) and Qdrant (scalar quantization with rescore: true) support int8 compression, cutting storage by ~75%.
• Automatic degradation: The engine cascades from Qdrant → sqlite-vec → FTS5 based on health checks, ensuring retrieval never fails completely.
• RRF unifies results: Reciprocal Rank Fusion with k=60 merges keyword and semantic rankings without arbitrary weights.
• Lazy migration: Quantization changes trigger needs_reindex flags, allowing background re-embedding without service interruption.

Frequently Asked Questions

What happens if the vector store fails to load?

If sqlite-vec fails to load or Qdrant becomes unreachable, OmniRoute automatically degrades to FTS5 keyword search. The engineStatus() endpoint reports the fallback, and retrieval continues using the memory_fts virtual table with MATCH queries and ranking.

How does OmniRoute handle the UUID primary key limitation in FTS5?

SQLite FTS5 requires an integer rowid, but OmniRoute uses UUID primary keys. Migration 023_fix_memory_fts_uuid.sql adds a memory_id column to the FTS table and creates triggers that map the UUID to the integer rowid, ensuring the virtual table stays synchronized with the main memories table on every insert, update, or delete.

Can I switch quantization modes without losing data?

Yes. Changing MEMORY_VEC_QUANTIZATION or the Qdrant quantization setting marks existing vectors with needs_reindex = 1 in memory_vec_meta. The system lazily re-embeds these entries during the next retrieval cycle, applying the new quantization format without blocking startup or requiring manual migration scripts.

What is the performance difference between FTS5 and vector search?

FTS5 provides millisecond-level exact matching with minimal memory overhead and no external dependencies. Vector search (sqlite-vec or Qdrant) requires embedding generation and distance calculations, offering superior semantic relevance at the cost of higher latency and memory usage, especially with Float32 vectors. Int8 quantization reduces the vector store memory footprint by approximately 4× while maintaining comparable recall through re-scoring mechanisms.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}