# How OmniRoute's Memory System Uses FTS5 and Vector Quantization: A Technical Deep Dive

> Explore OmniRoute's memory system combining FTS5 full-text search with vector quantization for efficient retrieval. Learn how it degrades gracefully to keyword search when needed.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-07-03

---

**OmniRoute implements a three-tier retrieval hierarchy that combines SQLite FTS5 full-text search with optional int8 vector quantization via sqlite-vec or Qdrant, automatically degrading to keyword search when vector stores are unavailable.**

OmniRoute stores conversational memories in a SQLite database and retrieves the most relevant ones using a hybrid architecture that balances precision, performance, and fault tolerance. The system is designed according to the source code in `diegosouzapw/OmniRoute` to prioritize exact keyword matches as a universal fallback while leveraging quantized vector embeddings for semantic similarity when available.

## Three-Tier Retrieval Architecture

The memory engine selects backends dynamically based on availability and configuration, forming a degradation chain from fastest to most resilient.

### Tier 0: FTS5 Keyword Search (Always Available)

The foundation of OmniRoute's memory system is a **SQLite FTS5 virtual table** that provides immediate text-based retrieval without external dependencies. During database migration [`022_add_memory_fts5.sql`](https://github.com/diegosouzapw/OmniRoute/blob/main/022_add_memory_fts5.sql), the system creates a virtual table named `memory_fts` indexing the `content` and `key` columns of the `memories` table.

A subsequent migration, [`023_fix_memory_fts_uuid.sql`](https://github.com/diegosouzapw/OmniRoute/blob/main/023_fix_memory_fts_uuid.sql), resolves the impedance mismatch between SQLite's integer `rowid` requirement and OmniRoute's UUID primary keys. It adds a `memory_id` column to the FTS table and establishes triggers that synchronize inserts, updates, and deletes between the main `memories` table and the FTS index.

In [`src/lib/memory/retrieval.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/retrieval.ts), the retrieval logic constructs queries using `memory_fts MATCH ?` syntax. For *exact* search strategies, or when vector stores fail to load, the engine executes a ranked FTS5 query ordered by the built-in `rank` function. If the FTS table returns no matches or is missing, the system falls back to a chronological scan using `ORDER BY created_at`.

The `engineStatus()` API explicitly reports `keyword.backend = "FTS5"` and `keyword.available = true`, confirming this tier is permanently active.

### Tier 1: sqlite-vec with Float32 and int8 Quantization

When the `sqlite-vec` extension loads successfully and an embedding source is configured (remote API, static Potion model, or local Transformers.js), the system activates vector search. The [`vectorStore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/vectorStore.ts) module creates a table `vec_memories` and populates it with embeddings.

**Vector quantization** is controlled via the `MEMORY_VEC_QUANTIZATION` environment variable. When set to `"int8"`, the insertion code in [`vectorStore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/vectorStore.ts) calls `vec_quantize_int8(?, 'unit')`, compressing vectors to 8-bit integers. This reduces storage footprint by approximately 4× compared to Float32, with a modest trade-off in recall accuracy.

Retrieval executes brute-force K-NN search using `SELECT * FROM vec_memories ORDER BY distance`, returning semantically similar memories based on cosine distance.

### Tier 2: Qdrant External Vector Database

If enabled in the Engine settings (`qdrantEnabled`), OmniRoute elevates Qdrant to the primary vector backend. The [`qdrant.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/qdrant.ts) module configures scalar quantization through the `normalizeQdrantConfig` function, reading the `qdrantQuantization` setting (`"none"`, `"int8"`, or `"binary"`).

For int8 mode, the driver constructs a scalar quantization configuration with `always_ram: true` and quantile `0.99`, enabling `rescore: true` to refine results using full-precision vectors during final ranking. This ensures that while storage is compressed, retrieval accuracy remains high through re-scoring.

## Hybrid Ranking with Reciprocal Rank Fusion

When both FTS5 and vector stores are available, OmniRoute employs **Reciprocal Rank Fusion (RRF)** to combine results. The implementation in [`retrieval.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/retrieval.ts) uses a default constant of `k = 60`, computing composite scores as:

```typescript
score = Σ (1 / (k + rank_i))

```

Where `rank_i` is the position of the document in each individual ranking list (FTS5 and vector). This produces a unified relevance score that balances keyword density with semantic similarity, eliminating the need to manually tune weighting parameters.

## Lazy Re-indexing and Quantization Migration

Changing quantization modes does not block startup. When `MEMORY_VEC_QUANTIZATION` changes, the system marks existing vectors with `needs_reindex = 1` in the `memory_vec_meta` table. The next retrieval operation triggers a lazy backfill that recomputes embeddings in the new format, allowing runtime configuration changes without downtime.

## Configuration and Code Examples

### Enable FTS5-Based Memory Retrieval

FTS5 is active by default once migrations run. Verify status via the API:

```bash
curl -X GET http://localhost:20128/api/memory/engine-status \
  -H "Authorization: Bearer $OMNIROUTE_KEY"

```

Expected response includes:

```json
{
  "keyword": {
    "backend": "FTS5",
    "available": true
  }
}

```

### Enable int8 Quantization with sqlite-vec

Set the environment variable before starting the server:

```bash
export MEMORY_VEC_QUANTIZATION=int8
npm start

```

This forces a re-index of `vec_memories`; the next retrieval lazily recomputes embeddings in int8 format.

### Configure Qdrant with int8 Quantization

```bash
curl -X PUT http://localhost:20128/api/settings/qdrant \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -d '{
    "enabled": true,
    "host": "qdrant.local",
    "port": 6333,
    "collection": "omniroute_memory",
    "quantization": "int8"
  }'

```

### Execute Hybrid Retrieval

```bash
curl -X POST http://localhost:20128/api/memory/retrieve-preview \
  -H "Authorization: Bearer $OMNIROUTE_KEY" \
  -d '{
    "strategy": "hybrid",
    "query": "TypeScript logging best practices",
    "maxTokens": 1500
  }'

```

The response includes `score`, `tier` (indicating FTS5 or vector origin), and the active quantization mode.

## Summary

- **FTS5 is the invariant backbone**: Implemented via [`022_add_memory_fts5.sql`](https://github.com/diegosouzapw/OmniRoute/blob/main/022_add_memory_fts5.sql) and [`023_fix_memory_fts_uuid.sql`](https://github.com/diegosouzapw/OmniRoute/blob/main/023_fix_memory_fts_uuid.sql), it provides keyword search with UUID-to-rowid mapping and automatic synchronization triggers.
- **Vector quantization reduces footprint**: Both sqlite-vec (`vec_quantize_int8`) and Qdrant (scalar quantization with `rescore: true`) support int8 compression, cutting storage by ~75%.
- **Automatic degradation**: The engine cascades from Qdrant → sqlite-vec → FTS5 based on health checks, ensuring retrieval never fails completely.
- **RRF unifies results**: Reciprocal Rank Fusion with `k=60` merges keyword and semantic rankings without arbitrary weights.
- **Lazy migration**: Quantization changes trigger `needs_reindex` flags, allowing background re-embedding without service interruption.

## Frequently Asked Questions

### What happens if the vector store fails to load?

If `sqlite-vec` fails to load or Qdrant becomes unreachable, OmniRoute automatically degrades to **FTS5 keyword search**. The `engineStatus()` endpoint reports the fallback, and retrieval continues using the `memory_fts` virtual table with `MATCH` queries and ranking.

### How does OmniRoute handle the UUID primary key limitation in FTS5?

SQLite FTS5 requires an integer `rowid`, but OmniRoute uses UUID primary keys. Migration [`023_fix_memory_fts_uuid.sql`](https://github.com/diegosouzapw/OmniRoute/blob/main/023_fix_memory_fts_uuid.sql) adds a `memory_id` column to the FTS table and creates triggers that map the UUID to the integer `rowid`, ensuring the virtual table stays synchronized with the main `memories` table on every insert, update, or delete.

### Can I switch quantization modes without losing data?

Yes. Changing `MEMORY_VEC_QUANTIZATION` or the Qdrant quantization setting marks existing vectors with `needs_reindex = 1` in `memory_vec_meta`. The system lazily re-embeds these entries during the next retrieval cycle, applying the new quantization format without blocking startup or requiring manual migration scripts.

### What is the performance difference between FTS5 and vector search?

FTS5 provides millisecond-level exact matching with minimal memory overhead and no external dependencies. Vector search (sqlite-vec or Qdrant) requires embedding generation and distance calculations, offering superior semantic relevance at the cost of higher latency and memory usage, especially with Float32 vectors. Int8 quantization reduces the vector store memory footprint by approximately 4× while maintaining comparable recall through re-scoring mechanisms.