How OmniRoute's Memory System Utilizes FTS5 Full-Text Search and Vector Quantization

OmniRoute implements a three-tier retrieval hierarchy that uses SQLite FTS5 for keyword search, sqlite-vec for semantic vector search, and optional Qdrant integration, with support for int8 quantization to reduce storage by approximately 75% while maintaining recall accuracy.

OmniRoute is an open-source conversational memory system that stores AI interaction histories in a SQLite database. According to the diegosouzapw/OmniRoute source code, the architecture combines traditional full-text indexing with modern vector quantization to deliver fast, relevant context retrieval across multiple search strategies.

The Three-Tier Retrieval Architecture

The retrieval engine operates on a cascading hierarchy that automatically selects the best available backend for each request:

• Tier 0 (FTS5): SQLite full-text search for exact keyword matching and universal fallback
• Tier 1 (sqlite-vec): Brute-force K-nearest neighbor search over Float32 or quantized embeddings
• Tier 2 (Qdrant): Optional external vector database for high-performance semantic search

Each tier serves specific query strategies and provides graceful degradation when higher tiers are unavailable.

Tier 0: FTS5 Full-Text Search Implementation

FTS5 serves as the foundation of OmniRoute's memory system, providing reliable keyword-based retrieval that is always available regardless of other configuration.

Database Schema and Migrations

The FTS5 virtual table is created during database migration. In src/lib/db/migrations/022_add_memory_fts5.sql, the system creates a virtual table named memory_fts indexed over the content and key columns of the memories table.

A subsequent migration in 023_fix_memory_fts_uuid.sql resolves the UUID primary key mapping to FTS5's required integer rowid. This migration adds a memory_id column and implements triggers that automatically synchronize the FTS5 index when memories are inserted, updated, or deleted.

Retrieval Logic

The retrieval code in src/lib/memory/retrieval.ts constructs FTS5 queries using the MATCH operator. For semantic or hybrid strategies, the system joins against memory_fts MATCH ? and orders results by FTS rank. If the join returns no rows or the table is missing, the engine automatically falls back to a chronological scan ordered by timestamp.

The engine status API confirms FTS5 availability at /api/memory/engine-status, reporting keyword.backend = "FTS5" and keyword.available = true as implemented in src/app/api/memory/engine-status/route.ts.

Tier 1: sqlite-vec Vector Storage

When the sqlite-vec extension loads successfully, OmniRoute enables semantic search over vector embeddings.

Vector Store Initialization

In src/lib/memory/vectorStore.ts, the system creates a vec_memories table and populates it with embeddings from the configured source—whether remote API, static Potion model, or local Transformers.js. The initialization checks the MEMORY_VEC_QUANTIZATION environment variable to determine the storage format.

K-NN Search Implementation

Retrieval performs a brute-force K-NN search using the query SELECT * FROM vec_memories ORDER BY distance. Results are combined with FTS5 output using Reciprocal Rank Fusion (RRF) with a default k value of 60 to produce the final hybrid ranking.

Tier 2: Qdrant External Vector Database

For production deployments requiring high-throughput semantic search, OmniRoute supports Qdrant as an optional external vector store.

Configuration and Quantization

The src/lib/memory/qdrant.ts file implements the Qdrant driver, reading quantization settings from the database key qdrantQuantization. When set to "int8", the driver configures scalar quantization with always_ram and quantile 0.99, enabling rescore: true to use full-precision vectors for final ranking.

If Qdrant is enabled but unhealthy, requests automatically fall back to sqlite-vec; if that fails, the system uses FTS5 as the final fallback.

Vector Quantization with int8

Both vector backends support optional int8 quantization, reducing vector storage size by approximately 75% with modest recall impact.

sqlite-vec Quantization

The MEMORY_VEC_QUANTIZATION environment variable controls the sqlite-vec mode. When set to "int8", insertion code in vectorStore.ts calls vec_quantize_int8(?, 'unit') to store vectors in compact 8-bit format.

Qdrant Quantization

Qdrant reads the qdrantQuantization setting (values: "none", "int8", or "binary"). When configured for int8, the system builds a scalar-quantization configuration that keeps vectors in RAM for fast access while rescoring with full precision.

Lazy Re-indexing

When quantization modes change, the system marks existing vectors for re-indexing by setting needs_reindex = 1 in memory_vec_meta. The next retrieval trigger lazily re-embeds these entries, ensuring the new format applies without blocking startup.

Configuration Examples

Enable Memory and FTS5

By default, memory is disabled. Enable it via the Settings API:

curl -X PUT http://localhost:20128/api/settings/memory \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -d '{"enabled":true,"maxTokens":2000}'

Configure int8 Quantization

Set the environment variable before starting OmniRoute:

export MEMORY_VEC_QUANTIZATION=int8
npm start

This triggers a full re-index of vec_memories on the next retrieval.

Enable Qdrant with Quantization

Configure Qdrant via the API:

curl -X PUT http://localhost:20128/api/settings/qdrant \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -d '{
        "enabled":true,
        "host":"qdrant.local",
        "port":6333,
        "collection":"omniroute_memory",
        "quantization":"int8"
      }'

Retrieve Memories with Hybrid Strategy

Preview ranked results:

curl -X POST http://localhost:20128/api/memory/retrieve-preview \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -d '{
        "strategy":"hybrid",
        "query":"TypeScript logging",
        "maxTokens":1500
      }'

The response includes score, tier (FTS5 or vector), and the active quantization mode.

Summary

• OmniRoute uses a three-tier hierarchy: FTS5 (always available), sqlite-vec (semantic), and Qdrant (external).
• FTS5 provides keyword search via SQLite virtual tables with automatic UUID-to-rowid mapping through migrations 022_add_memory_fts5.sql and 023_fix_memory_fts_uuid.sql.
• Vector quantization supports int8 storage via MEMORY_VEC_QUANTIZATION for sqlite-vec and qdrantQuantization for Qdrant, reducing storage by ~75%.
• Hybrid retrieval combines FTS5 and vector results using Reciprocal Rank Fusion (RRF) with k=60 as implemented in src/lib/memory/retrieval.ts.
• Automatic degradation falls back from Qdrant → sqlite-vec → FTS5 based on availability.
• Lazy re-indexing applies quantization changes without blocking startup by marking needs_reindex in memory_vec_meta.

Frequently Asked Questions

What is the difference between FTS5 and sqlite-vec in OmniRoute?

FTS5 is a SQLite virtual table extension optimized for full-text keyword search, always available as the fallback tier. sqlite-vec is a separate SQLite extension that performs brute-force K-NN search over floating-point or quantized vectors. FTS5 handles exact word matching while sqlite-vec handles semantic similarity.

How do I enable int8 quantization to reduce memory usage?

Set the environment variable MEMORY_VEC_QUANTIZATION=int8 before starting the application. For Qdrant, use the API to set quantization: "int8". The system will lazily re-index existing vectors on the next retrieval request, converting them to 8-bit integers without downtime.

What happens if Qdrant becomes unavailable?

The retrieval engine automatically degrades to the next available tier. If Qdrant is unhealthy, requests fall back to sqlite-vec. If sqlite-vec fails or is disabled, the system uses FTS5 full-text search. The /api/memory/engine-status endpoint reports current tier availability and fallback status.

Where does OmniRoute store the FTS5 index?

The FTS5 virtual table memory_fts is stored in the same SQLite database as the main application data. It is created by migrations 022_add_memory_fts5.sql and 023_fix_memory_fts_uuid.sql, which define the schema and automatic triggers to keep the index synchronized with the memories table.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}