OmniRoute Memory System for Conversational Context: Architecture and Implementation Guide

The OmniRoute Memory system provides persistent, searchable conversational context through a singleton MemoryManager that coordinates backend storage, vector retrieval, and typed decay to inject relevant memories into LLM prompts.

This deep dive explores how OmniRoute, an open-source LLM routing platform, implements long-term memory for AI applications. The system stores conversation history as typed memories that can be retrieved, summarized, and injected into prompts to maintain contextual continuity across chat sessions.

Core Architecture

The memory subsystem centers on a singleton orchestrator pattern that abstracts storage complexity while providing flexible backend options.

MemoryManager Singleton

At the heart of the system lies MemoryManager, defined in [src/lib/memory/manager.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/lib/memory/manager.ts). This singleton maintains a registry of backend implementations and handles primary-fallback routing.

The manager exposes two critical lifecycle methods:

• initialize() – Boots all registered backends, creates database tables, and loads embedding models.
• shutdown() – Gracefully closes connections and flushes pending writes.

By default, the primary backend ID is set to "sqlite", with fallback backends queried in sequence if the primary fails. This ensures high availability for conversational context even when external vector stores are unreachable.

Backend Interface and Implementations

All storage backends implement the MemoryBackend interface specified in [src/lib/memory/backend.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/lib/memory/backend.ts). This contract standardizes CRUD operations, health checks, and vector similarity search across implementations.

OmniRoute ships with multiple backend options:

• SQLite – Local persistent store with FTS5 full-text search, implemented in [src/lib/memory/sqliteBackend.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/lib/memory/sqliteBackend.ts).
• Qdrant – Remote vector database for semantic similarity with optional int8 quantization, found in [src/lib/memory/qdrant.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/lib/memory/qdrant.ts).
• Obsidian – Optional vault-based storage for personal knowledge management.

Memory Data Model

Memory entries follow a strict TypeScript interface defined in [src/lib/memory/types.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/lib/memory/types.ts). The Memory structure captures both content and metadata necessary for intelligent retrieval:

export interface Memory {
  uuid: string;                 // stable identifier
  type: MemoryType;             // FACTUAL | EPISODIC | SEMANTIC | PROCEDURAL
  key: string;                  // human-readable lookup key
  content: string;              // raw text or JSON payload
  metadata?: Record<string, any>;
  // ... timestamps, embedding vectors, decay counters, etc.
}

The four memory types serve distinct purposes in conversational context:

• FACTUAL – Persistent knowledge about users, preferences, or domain facts.
• EPISODIC – Specific conversation events that decay over time.
• SEMANTIC – Conceptual information requiring vector similarity matching.
• PROCEDURAL – System instructions or workflow steps.

The Four Memory Operations

The subsystem exposes four high-level operations, each implemented in dedicated modules under src/lib/memory/.

Extraction

The extraction module ([src/lib/memory/extraction.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/lib/memory/extraction.ts)) analyzes conversation history to identify salient information worth storing. It parses raw chat logs into structured Memory objects, automatically assigning types based on content classification.

Injection

Before sending prompts to an LLM, the injection module ([src/lib/memory/injection.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/lib/memory/injection.ts)) retrieves relevant memories and prepends them to the user message. This reduces token usage by surfacing prior context without resending entire conversation histories.

Clients can suppress injection for stateless calls using the x-omniroute-no-memory request header.

Retrieval

Vector and full-text search capabilities reside in [src/lib/memory/retrieval.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/lib/memory/retrieval.ts). The system queries SQLite FTS5 for keyword matches and Qdrant (via sqlite-vec) for semantic similarity, merging results using relevance scoring.

Summarization and Typed Decay

Long-term storage management is handled by [src/lib/memory/summarization.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/lib/memory/summarization.ts), which condenses older episodic memories to prevent storage bloat.

The typed decay feature ([src/lib/memory/typedDecay.ts](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/src/lib/memory/typedDecay.ts)) automatically purges stale EPISODIC entries while preserving FACTUAL, SEMANTIC, and PROCEDURAL memories. Enable this behavior by setting the MEMORY_TYPED_DECAY_ENABLED environment flag.

REST API and Configuration

All memory operations are exposed via REST endpoints under /api/memory/, documented in the OpenAPI specification at [docs/openapi.yaml](https://github.com/diegosouzapw/OmniRoute/blob/release/v3.8.50/docs/openapi.yaml).

Key endpoints include:

• GET /api/memory/health – Returns database connection status and vector index health.
• GET /api/memory/stats – Provides aggregate metrics including total entries, embedding coverage, and storage utilization.
• POST /api/memory – Creates or upserts memories, regenerating embeddings if a source is available.
• PUT /api/memory/{uuid} – Updates existing memory content, metadata, or type classification.

These routes are protected by requireManagementAuth middleware and respect the x-omniroute-no-memory header for stateless request overrides.

Implementation Examples

Registering Backends at Startup

Initialize the memory system by registering your preferred backends before the application begins accepting traffic:

import { MemoryManager } from '@/lib/memory/manager';
import { SQLiteBackend } from '@/lib/memory/sqliteBackend';

const manager = MemoryManager.getInstance();
manager.register(new SQLiteBackend());
await manager.initialize();   // boots DB, creates tables, loads embeddings

Injecting Context into Chat Requests

Enrich prompts with relevant historical context using the injection utility:

import { injectMemories } from '@/lib/memory/injection';

async function handleChat(prompt: string) {
  const enriched = await injectMemories({
    userPrompt: prompt,
    // optional filter – e.g. only FACTUAL memories about "billing"
    filter: { type: 'FACTUAL', key: /billing/i },
  });
  // enriched.prompt now contains the original text plus retrieved memories
  return callLLM(enriched.prompt);
}

Triggering Manual Decay Sweeps

For maintenance windows or cron jobs, manually execute typed decay to clean expired episodic data:

import { runTypedDecay } from '@/lib/memory/typedDecay';

await runTypedDecay({ dryRun: false }); // permanently deletes expired episodic memories

Summary

• Singleton Architecture – The MemoryManager in src/lib/memory/manager.ts coordinates multiple backends through a unified interface.
• Typed Storage – Memories are classified as FACTUAL, EPISODIC, SEMANTIC, or PROCEDURAL, enabling differentiated retention policies.
• Hybrid Retrieval – Combines SQLite FTS5 full-text search with Qdrant vector similarity for comprehensive context matching.
• Privacy-First Design – The system is opt-in by default, respects the x-omniroute-no-memory header, and never mutates payloads without explicit configuration.

Frequently Asked Questions

How does OmniRoute handle memory storage failures?

The MemoryManager implements primary-fallback routing where the default SQLite backend (primaryBackendId = "sqlite") serves as the persistent store. If the primary fails, the system automatically queries registered fallback backends in order. Additionally, the /api/memory/health endpoint exposes database connection status and vector index health for monitoring.

What is the difference between EPISODIC and FACTUAL memory types?

EPISODIC memories capture specific conversation events and are subject to typed decay, meaning they expire after a configurable period to prevent storage bloat. FACTUAL memories store persistent knowledge about users, preferences, or domain facts that remain indefinitely unless manually deleted. This distinction allows the system to preserve important knowledge while automatically cleaning transient chat history.

Can I disable memory injection for specific requests?

Yes. Clients can send the x-omniroute-no-memory header to suppress memory injection for stateless API calls. This ensures that sensitive or one-off queries do not retrieve historical context, effectively treating the request as memory-less while still allowing the backend to store new memories if configured to do so.

How do I enable automatic cleanup of old conversation memories?

Enable typed decay by setting the MEMORY_TYPED_DECAY_ENABLED environment variable. This activates the decay sweep logic in src/lib/memory/typedDecay.ts, which periodically deletes expired EPISODIC memories while preserving FACTUAL, SEMANTIC, and PROCEDURAL entries. You can also manually trigger cleanup using runTypedDecay({ dryRun: false }) in scheduled jobs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}