How OmniRoute's Memory System Manages Persistent Context Across Sessions

OmniRoute uses a hybrid architecture combining an in-memory LRU cache with a SQLite-backed persistent store to maintain conversational context across server restarts and user sessions.

The diegosouzapw/OmniRoute repository implements a sophisticated memory subsystem designed to solve the challenge of maintaining stateful conversations in stateless HTTP environments. By blending high-speed in-memory access with durable SQLite storage, the system ensures that user context survives process crashes and deployments while delivering sub-millisecond read performance.

Hybrid Architecture: SQLite Persistence with In-Memory Acceleration

OmniRoute's memory system stores all conversational data permanently in SQLite while using a Map-based LRU cache for hot data access. This dual-layer approach guarantees durability without sacrificing latency.

Persistent Storage Layer

The core persistence logic resides in src/lib/memory/store.ts, which handles CRUD operations against the SQLite memory table. Every memory entry receives a unique identifier and maintains foreign key relationships to API keys and session IDs, enabling multi-tenant isolation. When the server restarts, this table remains intact, serving as the source of truth for all historical context.

In-Memory Cache Layer

The src/lib/memory/cache.ts module implements the performance optimization layer. It uses a composite cache key format combining apiKeyId, sessionId, and memoryId to isolate tenant data while maximizing hit rates. Recent entries populate this cache immediately after database insertion, ensuring that active conversations never hit disk for subsequent retrievals.

Memory Creation and Session Binding

New memories enter the system through a validated injection pipeline defined in src/lib/memory/injection.ts. When a request arrives containing the X-OmniRoute-Session-Id header, the middleware triggers the creation flow:

1. Validation: The payload undergoes strict schema validation using Zod definitions in src/lib/memory/schemas.ts, ensuring type safety for content, metadata, and memory classifications.
2. Persistence: Valid entries write immediately to SQLite via the store module.
3. Cache Promotion: The same row instantiates in the in-memory cache, making it available for immediate retrieval without a database round-trip.

Retrieval Strategy: Cache-First with Database Fallback

The retrieval logic in src/lib/memory/retrieval.ts implements a resilient lookup pattern optimized for speed. When fetching prior context, the system first queries the in-memory Map for the composite key. A cache hit returns the entry in sub-millisecond time. On a cache miss, the system falls back to SQLite, loads the row, and promotes it back into the cache to accelerate future accesses for that session.

This pattern ensures that after a server restart, the first request for any memory incurs the SQLite latency penalty, but all subsequent accesses for that conversation remain in-memory until the entry ages out of the LRU window.

Semantic Search via Vector Embeddings

Beyond exact retrieval, OmniRoute supports semantic similarity search through its vector store implementation in src/lib/memory/vectorStore.ts. The system generates embeddings using either local transformers (src/lib/memory/embedding/transformersLocal.ts) or remote services (src/lib/memory/embedding/remote.ts), storing vectors in SQLite via the sqlite-vec extension.

By default, semantic queries return the top 20 nearest neighbors (MEMORY_VEC_TOP_K=20), enabling the system to surface contextually relevant memories even when keyword matching fails.

Automated Maintenance with Typed Decay

To prevent unbounded storage growth, the optional typed decay system in src/lib/memory/typedDecay.ts prunes stale episodic memories. When enabled via MEMORY_TYPED_DECAY_ENABLED=true, a background sweep runs at intervals defined by MEMORY_TYPED_DECAY_SWEEP_INTERVAL, deleting entries older than MEMORY_TYPED_DECAY_EPISODIC_DAYS from both the SQLite table and the in-memory cache. This cleanup targets only episodic memory types, preserving critical procedural or semantic knowledge.

Crash Recovery and Session Continuity

The architecture guarantees crash recovery through its storage hierarchy. When the Node.js process restarts, the in-memory cache initializes as empty, but the SQLite database retains all unexpired rows. The first retrieval request for any session automatically repopulates the cache from disk, restoring the previous conversational state transparently to the user. This design eliminates the need for complex snapshotting or external Redis dependencies while maintaining zero-data-loss guarantees.

REST API for Memory Operations

The memory subsystem exposes full CRUD capabilities through the /api/memory/* REST endpoints documented in docs/reference/API_REFERENCE.md. These endpoints support creation, retrieval, semantic search, and targeted deletion of memory entries scoped to specific sessions.

Creating a Memory Entry

fetch('/api/memory', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'X-OmniRoute-Session-Id': 'session-123',
    'Authorization': 'Bearer <api-key>'
  },
  body: JSON.stringify({
    content: 'User asked about travel to Paris',
    type: 'EPISODIC',
    metadata: { intent: 'travel' }
  })
})
.then(r => r.json())
.then(console.log);

Retrieving Recent Context

fetch('/api/memory?sessionId=session-123&limit=10', {
  headers: { Authorization: 'Bearer <api-key>' }
})
.then(r => r.json())
.then(memories => {
  console.log('Recent memories:', memories);
});

Semantic Vector Search

fetch('/api/memory/search?q=Paris', {
  headers: { Authorization: 'Bearer <api-key>' }
})
.then(r => r.json())
.then(results => console.log('Search results:', results));

Clearing Episodic Memories

fetch('/api/memory/clear', {
  method: 'POST',
  headers: { 
    'Content-Type': 'application/json', 
    'Authorization': 'Bearer <api-key>' 
  },
  body: JSON.stringify({ 
    sessionId: 'session-123', 
    type: 'EPISODIC' 
  })
});

Summary

• Hybrid Storage: OmniRoute combines SQLite durability with an in-memory LRU cache (src/lib/memory/cache.ts) to balance persistence with performance.
• Automatic Recovery: Server restarts trigger transparent cache repopulation from src/lib/memory/store.ts, ensuring no conversational context is lost.
• Semantic Capabilities: The vector store in src/lib/memory/vectorStore.ts enables similarity search via sqlite-vec embeddings with configurable top-k results.
• Schema Validation: All memory inputs validate against Zod schemas in src/lib/memory/schemas.ts before reaching the database layer.
• Configurable Lifecycle: Optional typed decay (`src/lib/memory/typedDecay.ts) automatically prunes old episodic memories based on age thresholds.

Frequently Asked Questions

How does OmniRoute handle memory persistence after a server crash?

According to the diegosouzapw/OmniRoute source code, the system stores all memories in a SQLite database managed by src/lib/memory/store.ts. When the server restarts, the in-memory cache starts empty, but the SQLite table retains all data. The first retrieval operation for any session loads the data from disk and repopulates the cache, ensuring conversational continuity without manual intervention.

What is the cache key structure for OmniRoute's memory system?

The LRU cache in src/lib/memory/cache.ts uses a composite key format combining apiKeyId + sessionId + memoryId. This structure ensures tenant isolation while allowing efficient lookup of specific memories within a user's session history.

Can OmniRoute perform semantic search on historical conversations?

Yes. The system generates vector embeddings using either local transformers (src/lib/memory/embedding/transformersLocal.ts) or remote services (src/lib/memory/embedding/remote.ts), storing them in SQLite via the sqlite-vec extension. The src/lib/memory/vectorStore.ts module handles nearest-neighbor queries with a default MEMORY_VEC_TOP_K of 20 results.

How does typed decay work in OmniRoute's memory management?

Typed decay is an optional cleanup mechanism in src/lib/memory/typedDecay.ts that removes stale episodic memories. When MEMORY_TYPED_DECAY_ENABLED is set to true, the system runs sweeps at intervals defined by MEMORY_TYPED_DECAY_SWEEP_INTERVAL, deleting entries older than MEMORY_TYPED_DECAY_EPISODIC_DAYS from both the SQLite database and the in-memory cache.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}