Understanding OmniRoute's Memory System: FTS5, Vector Quantization, and Typed Memory Decay

OmniRoute stores conversational memory in SQLite with high-dimensional embeddings, using FTSS (Full-Text Search + Vector) hybrid retrieval, int8 quantization for storage efficiency, and configurable typed decay rules to prune stale memories based on access patterns and TTL.

OmniRoute is an open-source conversational agent framework that persists memory in a SQLite database enriched with high-dimensional embeddings. Understanding OmniRoute's memory system requires examining three interconnected mechanisms: the FTSS hybrid retrieval architecture that combines full-text and vector search, aggressive int8 quantization to reduce storage footprint, and a sophisticated typed decay system that manages memory lifecycle by category.

The FTSS Architecture: Storing Embeddings in SQLite

Virtual Tables and the Core Schema

The memory subsystem centers on a memories table that stores JSON payloads and metadata including type, accessCount, and timestamps. Separately, the system maintains a vec_memories virtual table implemented via SQLite-vec, which houses the high-dimensional embedding vectors. This separation allows pure vector similarity search against vec_memories while keeping metadata operations efficient against the primary table, as implemented in src/lib/memory/vectorStore.ts.

The Embedding Pipeline

When the embedding pipeline in src/lib/memory/embedding/ computes a vector for new content, VectorStore.upsertVector() handles persistence. The method looks up the rowid of the memory record, deletes any existing vector row, and inserts the new embedding using an expression built by vecValueExpr(q). This expression dynamically adapts to the current quantization mode, ensuring the stored column type always matches the signature recorded in memory_vec_meta.

Vector Quantization and Storage Efficiency

Configuring int8 Quantization Mode

By default, OmniRoute stores vectors as 32-bit floats (FLOAT[N]). However, when MEMORY_VEC_QUANTIZATION is set to "int8", the system switches to int8 quantization via the vec_quantize_int8(?, 'unit') SQL helper. This reduces disk usage by approximately 4× while maintaining search quality.

In src/lib/memory/vectorStore.ts, the VecQuantization type defines the supported modes, while requestedVecQuantization() and storedVecQuantization() helpers determine the active configuration. The vecColumnType() function builds the appropriate column definition string based on these settings.

Signature Changes and Re-indexing

The effective signature combines the resolution and model identifier (e.g., "openai-embed-3") with a quantization suffix (:int8 when enabled). When VectorStoreImpl.resetForSignature() detects a signature change, it drops and recreates vec_memories with the new column type and marks all memories as needing re-index by setting needsReindex=1. Re-indexing occurs lazily when vectors are next upserted.

Typed Memory Decay Lifecycle Management

TTL and Immunity Rules

The decay system in src/lib/memory/typedDecay.ts categorizes memories into types: episodic, factual, procedural, and semantic. Each type has a configurable TTL (days) defined in DEFAULT_TTL_DAYS_BY_TYPE. A null TTL grants permanent immunity to that type.

Dual immunity protects frequently-used memories. The isAccessImmune() function checks if a memory's accessCount exceeds the accessImmunityThreshold (configured via MEMORY_TYPED_DECAY_ACCESS_IMMUNITY). The isTypeImmune() function evaluates the type-specific TTL. Only memories failing both checks are eligible for deletion.

The Sweep Process

The sweepDecayedMemories() function implements the actual cleanup. It first checks the master switch MEMORY_TYPED_DECAY_ENABLED, returning early with skippedDisabled:true if disabled. The sweep queries candidates via listMemoriesForDecay(), respecting the SWEEP_SCAN_CAP limit. For each candidate, isMemoryDecayed() calculates the deadline based on creation time and type-specific TTL. Expired memories trigger deleteMemory(), which removes the row from memories and calls deleteVector() to maintain consistency across tables.

Activation and Scheduling

The startMemoryDecaySweep() function creates a periodic timer only when both the master switch is enabled and MEMORY_TYPED_DECAY_SWEEP_INTERVAL (seconds) is positive. The resolveSweepIntervalMs() helper converts this to milliseconds for the timer.

Hybrid Search Implementation

Pure Vector Search

The searchVector() method issues a MATCH query against the vec_memories virtual table. It uses vecValueExpr(q) to bind the query vector, ensuring the database engine applies quantization if the table operates in int8 mode. This provides sub-millisecond nearest-neighbor lookups.

Reciprocal Rank Fusion (RRF)

The searchHybrid() method combines vector results with full-text search against memory_fts using Reciprocal Rank Fusion. The implementation uses a CTE-based SQL pipeline that merges result sets with the formula 1 / (rank + k), where k defaults to 60 and is configurable via MEMORY_RRF_K. This hybrid approach balances semantic similarity with lexical matching.

Configuration and Usage Examples

Enable int8 quantization and initialize the vector store:

process.env.MEMORY_VEC_QUANTIZATION = "int8";
const vecStore = getVectorStore();
if (vecStore) {
  await vecStore.ensureReady({ dimensions: 384, signature: "openai-embed-3" });
}

Insert a memory with its embedding:

await vecStore!.upsertVector(memoryId, new Float32Array([0.12, -0.34, /* ... */]));

Configure and start typed decay:

process.env.MEMORY_TYPED_DECAY_ENABLED = "true";
process.env.MEMORY_TYPED_DECAY_EPISODIC_DAYS = "14";
process.env.MEMORY_TYPED_DECAY_ACCESS_IMMUNITY = "5";

// Start periodic sweep (every hour)
startMemoryDecaySweep(3600);

Execute a dry-run sweep:

const result = await sweepDecayedMemories({ dryRun: true });
console.log(`Would delete ${result.decayed} memories`);

Perform hybrid search:

const hits = await vecStore!.searchHybrid(
  queryEmbedding,
  "how to reset my password",
  20,
  apiKeyId
);

Summary

• OmniRoute persists conversational memory in SQLite using a dual-table architecture: memories for metadata and vec_memories for embeddings.
• Vector quantization reduces storage by 75% when int8 mode is enabled via MEMORY_VEC_QUANTIZATION, with automatic re-indexing when signatures change.
• Typed decay provides configurable TTLs per memory type (episodic, factual, procedural, semantic) with dual immunity protection based on access count.
• FTSS hybrid search combines vector similarity with full-text search using Reciprocal Rank Fusion, defaulting to RRF_K=60.
• The sweep process is opt-in via MEMORY_TYPED_DECAY_ENABLED and respects SWEEP_SCAN_CAP to prevent resource exhaustion.

Frequently Asked Questions

What is FTSS in OmniRoute's memory system?

FTSS stands for Full-Text Search plus Vector Search, referring to OmniRoute's hybrid retrieval architecture. The system maintains separate virtual tables for vectors (vec_memories) and full-text content (memory_fts), combining results via Reciprocal Rank Fusion to balance semantic similarity with keyword matching.

How does int8 quantization affect search accuracy?

According to the vectorStore.ts implementation, int8 quantization uses the vec_quantize_int8(?, 'unit') SQL helper to normalize and quantize 32-bit floats into 8-bit integers. This reduces disk I/O and storage by approximately 4× while the SQLite-vec engine maintains search quality through the quantization algorithm.

Why does changing the quantization mode require re-indexing?

When MEMORY_VEC_QUANTIZATION changes, the column type in vec_memories must switch between FLOAT[N] and int8[N]. The resetForSignature() method drops and recreates the virtual table with the new schema, marking existing memories with needsReindex=1 so they are re-inserted with the correct format on next access.

How can I prevent important memories from being deleted by the decay sweep?

Memories become immune to decay through two mechanisms: type-based immunity (setting the TTL to null for that memory type) or access-based immunity (configuring MEMORY_TYPED_DECAY_ACCESS_IMMUNITY so memories accessed frequently survive the sweep). The isAccessImmune() and isTypeImmune() functions in typedDecay.ts enforce these rules.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}