# What Memory System Does OmniRoute Use? A Deep Dive into the Hybrid Architecture

> Discover OmniRoute's hybrid memory system featuring an LRU cache, SQLite for durability, and optional vector stores for advanced search capabilities. Learn more now.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: deep-dive
- Published: 2026-08-27

---

**OmniRoute uses a hybrid memory system that combines an in-process LRU cache for sub-millisecond access, durable SQLite storage for crash recovery, and optional vector stores for semantic search.**

The memory subsystem in the `diegosouzapw/OmniRoute` repository is engineered for AI applications requiring both speed and persistence. This article examines how the architecture layers hot-path caching, relational durability, and vector indexing to balance performance with data safety.

## The Hybrid Architecture Overview

OmniRoute’s memory stack is not a single database but a coordinated pipeline. The system routes every memory operation through [`src/lib/memory/store.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/store.ts), which acts as a backend abstraction layer. This design allows the `MemoryManager` façade in [`src/lib/memory/manager.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/manager.ts) to seamlessly switch between in-memory, SQLite, or external Qdrant backends without changing the public API.

## The Hot-Path Cache Layer

At the core of OmniRoute’s speed is a process-wide `Map` structure that serves as an LRU-by-creation cache.

### LRU-by-Creation with MAX_MEMORY_ENTRIES

The `MemoryManager` maintains a ceiling of **2000 entries** defined by the constant `MAX_MEMORY_ENTRIES` in [`src/lib/memory/manager.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/manager.ts). This cache stores recent memories directly in RAM, ensuring that conversational context and frequently accessed knowledge return in sub-millisecond timeframes. When the cache reaches capacity, older entries are evicted based on creation time, not access patterns, providing predictable memory usage for long-running processes.

## Durable Persistence with SQLite

To prevent data loss across process restarts, every write operation synchronizes to a local SQLite database.

### The sqliteBackend.ts Implementation

The file [`src/lib/memory/sqliteBackend.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/sqliteBackend.ts) implements the durable layer, persisting memories to a table named `memory`. This backend handles CRUD operations and schema management, making conversation histories searchable from the built-in dashboard and exportable for analysis. Because SQLite is embedded, the system requires no external database server for standard deployments.

## Retrieval Strategy

OmniRoute optimizes read performance through a tiered lookup strategy that minimizes database queries.

### Cache-First with Read-Through Fallback

The retrieval logic in [`src/lib/memory/retrieval.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/retrieval.ts) implements a strict cache-first policy. Lookups begin by querying the in-memory `Map`; on a cache miss, the system falls back to SQLite. Crucially, any row fetched from the database is immediately promoted back into the in-memory cache, warming it for subsequent requests. This read-through pattern ensures that once a memory is accessed, it remains fast for the remainder of the session.

## Vector Search and Embeddings

Beyond exact-match retrieval, OmniRoute supports semantic similarity search through vector embeddings.

### sqlite-vec Extension

By default, vector indexing uses the `sqlite-vec` extension implemented in [`src/lib/memory/vectorStore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/vectorStore.ts). This approach stores embedding vectors within the same SQLite file as the raw memory content, eliminating the need for separate vector infrastructure in small-to-medium deployments. When embeddings are enabled, the system automatically indexes new memories and performs cosine-similarity searches against the vector table.

### Optional Qdrant Integration

For workloads exceeding SQLite’s scalability limits, [`src/lib/memory/qdrant.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/qdrant.ts) provides an optional Qdrant sidecar. By toggling the `qdrantEnabled` configuration, operators can offload vector storage and ANN (Approximate Nearest Neighbor) search to a dedicated vector database while retaining SQLite for metadata and raw content storage.

## Memory Lifecycle Management

Long-running agents risk accumulating stale conversational data. OmniRoute addresses this through configurable decay policies.

### Automatic Pruning with Typed Decay

The module [`src/lib/memory/typedDecay.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/typedDecay.ts) implements time-to-live (TTL) pruning for episodic memories. Administrators can configure `MEMORY_TYPED_DECAY_*` settings to automatically delete memories with specific tags after a defined duration. This prevents context window pollution without manual database maintenance.

## Working with the Memory System

The following TypeScript examples demonstrate the public API surface exposed through [`src/lib/memory/manager.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/manager.ts):

```typescript
// 1️⃣ Initialise the memory manager (singleton)
import { getMemoryManager } from '@/lib/memory/manager';

// 2️⃣ Add a memory entry (automatically persisted & cached)
await getMemoryManager().add({
  role: 'assistant',
  content: 'The capital of France is Paris.',
  tags: ['knowledge'],
  // optional embedding request – uses the configured provider
  embed: true,
});

// 3️⃣ Perform a similarity search (vector‑based if enabled)
const results = await getMemoryManager().search({
  query: 'What is the capital of France?',
  topK: 5,               // return the 5 most similar memories
  filterTags: ['knowledge'],
});

// 4️⃣ Delete a specific memory by its internal ID
await getMemoryManager().delete(memoryId);

// 5️⃣ Clear the entire store (both cache and SQLite)
await getMemoryManager().clear();

```

All calls route through [`src/lib/memory/store.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/store.ts), which dispatches to the appropriate backend (SQLite, Qdrant, or an in-memory fake for test environments).

## Summary

- **Hybrid layering**: OmniRoute combines a 2000-entry in-memory LRU cache, SQLite persistence, and optional vector stores to balance speed with durability.
- **Cache-first retrieval**: The [`retrieval.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/retrieval.ts) module implements read-through logic, promoting SQLite results into RAM to minimize latency on repeated access.
- **Flexible vector backends**: Default `sqlite-vec` integration keeps deployments simple, while optional Qdrant support scales to larger datasets.
- **Lifecycle controls**: Typed decay policies in [`typedDecay.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/typedDecay.ts) automatically prune stale episodic memories based on configurable TTLs.

## Frequently Asked Questions

### Does OmniRoute require an external database to run?

No. OmniRoute functions entirely with embedded SQLite storage via [`sqliteBackend.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/sqliteBackend.ts) and the `sqlite-vec` extension for vectors. External Qdrant is strictly optional and only recommended for high-volume semantic search workloads.

### How many memories can OmniRoute cache in RAM?

The hot-path cache is capped at **2000 entries** as defined by `MAX_MEMORY_ENTRIES` in [`src/lib/memory/manager.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/manager.ts). Once this limit is reached, the oldest memories by creation time are evicted to make room for new entries.

### What happens to memories when the process restarts?

All memories persist to the SQLite `memory` table. On restart, the cache begins empty, but the read-through fallback in [`src/lib/memory/retrieval.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/retrieval.ts) repopulates it as the agent revisits previous contexts, ensuring no data loss between sessions.

### Can I disable vector search if I don't need semantic retrieval?

Yes. Vector indexing is opt-in via the `embed` flag on individual memory entries or global configuration. When disabled, the system skips [`vectorStore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/vectorStore.ts) operations and performs traditional tag-based or exact-text filtering through SQLite alone.