What Memory System Does OmniRoute Use? A Deep Dive into the Hybrid Architecture

OmniRoute uses a hybrid memory system that combines an in-process LRU cache for sub-millisecond access, durable SQLite storage for crash recovery, and optional vector stores for semantic search.

The memory subsystem in the diegosouzapw/OmniRoute repository is engineered for AI applications requiring both speed and persistence. This article examines how the architecture layers hot-path caching, relational durability, and vector indexing to balance performance with data safety.

The Hybrid Architecture Overview

OmniRoute’s memory stack is not a single database but a coordinated pipeline. The system routes every memory operation through src/lib/memory/store.ts, which acts as a backend abstraction layer. This design allows the MemoryManager façade in src/lib/memory/manager.ts to seamlessly switch between in-memory, SQLite, or external Qdrant backends without changing the public API.

The Hot-Path Cache Layer

At the core of OmniRoute’s speed is a process-wide Map structure that serves as an LRU-by-creation cache.

LRU-by-Creation with MAX_MEMORY_ENTRIES

The MemoryManager maintains a ceiling of 2000 entries defined by the constant MAX_MEMORY_ENTRIES in src/lib/memory/manager.ts. This cache stores recent memories directly in RAM, ensuring that conversational context and frequently accessed knowledge return in sub-millisecond timeframes. When the cache reaches capacity, older entries are evicted based on creation time, not access patterns, providing predictable memory usage for long-running processes.

Durable Persistence with SQLite

To prevent data loss across process restarts, every write operation synchronizes to a local SQLite database.

The sqliteBackend.ts Implementation

The file src/lib/memory/sqliteBackend.ts implements the durable layer, persisting memories to a table named memory. This backend handles CRUD operations and schema management, making conversation histories searchable from the built-in dashboard and exportable for analysis. Because SQLite is embedded, the system requires no external database server for standard deployments.

Retrieval Strategy

OmniRoute optimizes read performance through a tiered lookup strategy that minimizes database queries.

Cache-First with Read-Through Fallback

The retrieval logic in src/lib/memory/retrieval.ts implements a strict cache-first policy. Lookups begin by querying the in-memory Map; on a cache miss, the system falls back to SQLite. Crucially, any row fetched from the database is immediately promoted back into the in-memory cache, warming it for subsequent requests. This read-through pattern ensures that once a memory is accessed, it remains fast for the remainder of the session.

Vector Search and Embeddings

Beyond exact-match retrieval, OmniRoute supports semantic similarity search through vector embeddings.

sqlite-vec Extension

By default, vector indexing uses the sqlite-vec extension implemented in src/lib/memory/vectorStore.ts. This approach stores embedding vectors within the same SQLite file as the raw memory content, eliminating the need for separate vector infrastructure in small-to-medium deployments. When embeddings are enabled, the system automatically indexes new memories and performs cosine-similarity searches against the vector table.

Optional Qdrant Integration

For workloads exceeding SQLite’s scalability limits, src/lib/memory/qdrant.ts provides an optional Qdrant sidecar. By toggling the qdrantEnabled configuration, operators can offload vector storage and ANN (Approximate Nearest Neighbor) search to a dedicated vector database while retaining SQLite for metadata and raw content storage.

Memory Lifecycle Management

Long-running agents risk accumulating stale conversational data. OmniRoute addresses this through configurable decay policies.

Automatic Pruning with Typed Decay

The module src/lib/memory/typedDecay.ts implements time-to-live (TTL) pruning for episodic memories. Administrators can configure MEMORY_TYPED_DECAY_* settings to automatically delete memories with specific tags after a defined duration. This prevents context window pollution without manual database maintenance.

Working with the Memory System

The following TypeScript examples demonstrate the public API surface exposed through src/lib/memory/manager.ts:

// 1️⃣ Initialise the memory manager (singleton)
import { getMemoryManager } from '@/lib/memory/manager';

// 2️⃣ Add a memory entry (automatically persisted & cached)
await getMemoryManager().add({
  role: 'assistant',
  content: 'The capital of France is Paris.',
  tags: ['knowledge'],
  // optional embedding request – uses the configured provider
  embed: true,
});

// 3️⃣ Perform a similarity search (vector‑based if enabled)
const results = await getMemoryManager().search({
  query: 'What is the capital of France?',
  topK: 5,               // return the 5 most similar memories
  filterTags: ['knowledge'],
});

// 4️⃣ Delete a specific memory by its internal ID
await getMemoryManager().delete(memoryId);

// 5️⃣ Clear the entire store (both cache and SQLite)
await getMemoryManager().clear();

All calls route through src/lib/memory/store.ts, which dispatches to the appropriate backend (SQLite, Qdrant, or an in-memory fake for test environments).

Summary

• Hybrid layering: OmniRoute combines a 2000-entry in-memory LRU cache, SQLite persistence, and optional vector stores to balance speed with durability.
• Cache-first retrieval: The retrieval.ts module implements read-through logic, promoting SQLite results into RAM to minimize latency on repeated access.
• Flexible vector backends: Default sqlite-vec integration keeps deployments simple, while optional Qdrant support scales to larger datasets.
• Lifecycle controls: Typed decay policies in typedDecay.ts automatically prune stale episodic memories based on configurable TTLs.

Frequently Asked Questions

Does OmniRoute require an external database to run?

No. OmniRoute functions entirely with embedded SQLite storage via sqliteBackend.ts and the sqlite-vec extension for vectors. External Qdrant is strictly optional and only recommended for high-volume semantic search workloads.

How many memories can OmniRoute cache in RAM?

The hot-path cache is capped at 2000 entries as defined by MAX_MEMORY_ENTRIES in src/lib/memory/manager.ts. Once this limit is reached, the oldest memories by creation time are evicted to make room for new entries.

What happens to memories when the process restarts?

All memories persist to the SQLite memory table. On restart, the cache begins empty, but the read-through fallback in src/lib/memory/retrieval.ts repopulates it as the agent revisits previous contexts, ensuring no data loss between sessions.

Can I disable vector search if I don't need semantic retrieval?

Yes. Vector indexing is opt-in via the embed flag on individual memory entries or global configuration. When disabled, the system skips vectorStore.ts operations and performs traditional tag-based or exact-text filtering through SQLite alone.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}