# How the Memory System in OmniRoute Persists Conversational Context: Extraction, Injection, Retrieval, and Summarization

> Discover how OmniRoute's memory system uses extraction, injection, retrieval, and summarization to persist conversational context in LLM interactions. Learn more.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: internals
- Published: 2026-07-30

---

**The memory system in OmniRoute persists conversational context by extracting factual statements from LLM responses, storing them in SQLite, retrieving relevant memories via hybrid full-text and vector search, injecting them into subsequent prompts, and automatically summarizing older entries to enforce token limits.**

OmniRoute, the open-source AI routing gateway by diegosouzapw, implements a full-stack persistent memory layer that maintains state across stateless chat completions. This system operates through four distinct stages—extraction, injection, retrieval, and summarization—ensuring that long-running conversations retain critical context without overwhelming the LLM's context window.

## The Four Stages of OmniRoute's Memory System

### Extraction: Capturing Facts from LLM Responses

The **extraction** stage identifies and preserves factual information from every LLM response. In [`src/lib/memory/extraction.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/extraction.ts), the async `extractFacts()` pipeline processes incoming text through `extractFactsFromText()`, generating `ExtractedFact` objects that represent discrete pieces of information.

Each extracted fact becomes a persistent record with `MemoryType.FACTUAL` (or `SEMANTIC` for embedding vectors) written directly to SQLite. This stage runs automatically after the response streams back to the client, ensuring zero latency impact on the chat experience.

Key implementation details:
- Facts are extracted based on grammatical patterns and confidence scoring
- Metadata includes session ID, API key association, and timestamps
- Records link to the specific conversation thread for precise retrieval

### Injection: Inserting Context into Prompts

**Injection** determines whether and how to insert stored memories into new requests. The [`src/lib/memory/injection.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/injection.ts) module defines `shouldInjectMemory()`, which checks for the `x-omniroute-no-memory` header or the `noMemory` request flag to conditionally bypass memory loading.

When injection is enabled, `injectMemory()` executes `retrieveMemories()` to fetch relevant context, then formats the results using `formatMemoryContext()`. These memories are concatenated into the system message or appended to the user prompt before the request routes to the upstream provider.

The system supports flexible scoping:
- **Session-based**: Retrieves memories from the current `sessionId`
- **API-key based**: Shares memory across all sessions using the same key
- **Custom query**: Filters by specific content patterns or memory types

### Retrieval: Hybrid Search for Relevant Memories

**Retrieval** locates the most pertinent historical context using a hybrid approach defined in [`src/lib/memory/retrieval.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/retrieval.ts). The `retrieveMemories()` function combines SQLite's native **FTS5** full-text search with optional **vector similarity** searches through the built-in [`vectorStore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/vectorStore.ts) implementation.

Relevance scoring occurs in [`retrieval/scoring.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/retrieval/scoring.ts) via `getRelevanceScore()`, which ranks results by semantic proximity and recency. The system calculates similarity scores against the current conversation embedding, ensuring that injected memories remain contextually appropriate rather than merely keyword-matched.

The retrieval layer supports preview functionality through `retrievePreview()`, allowing administrators to audit which memories would be injected for a given query without executing the full pipeline.

### Summarization: Compressing Historical Context

**Summarization** prevents memory bloat by collapsing older or voluminous memory piles into dense summaries. The [`src/lib/memory/summarization.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/summarization.ts) module provides `summarizeMemories()` and `summarizeMemoriesOlderThan()`, which batch-process stale entries through the LLM using specialized condensation prompts.

When triggered, the function:
1. Selects memories exceeding the age or count thresholds defined in [`src/lib/memory/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/settings.ts) (controlled by `MEMORY_MAX_SIZE` and `MEMORY_TYPED_DECAY_*` environment variables)
2. Sends the batch to the LLM with a summarization prompt
3. Stores the resulting compact text as a new memory with `MemoryType.SUMMARY`
4. Archives or removes the original detailed entries

This process maintains conversational coherence while keeping token usage bounded for the injection stage.

## How the Memory Pipeline Works in Practice

The complete request-response cycle demonstrates how these stages interact:

1. **Request arrival**: Middleware checks `shouldInjectMemory()` for the incoming `/api/v1/chat/completions` request
2. **Context loading**: If enabled, `retrieveMemories()` fetches relevant facts using the session ID and query parameters, scored by `getRelevanceScore()`
3. **Prompt assembly**: `formatMemoryContext()` converts memories into formatted strings merged into the payload
4. **Upstream routing**: The enriched request routes to the selected provider (OpenAI, Anthropic, etc.)
5. **Response processing**: After streaming the response to the client, `extractFacts()` parses the LLM output via `extractFactsFromText()`
6. **Persistence**: Each extracted fact writes to the `memories` table as `MemoryType.FACTUAL`
7. **Maintenance**: Background jobs periodically execute `summarizeMemoriesOlderThan()` to compress entries older than the configured threshold

## Configuration and Storage Architecture

All memory operations persist to **SQLite** with optional **Qdrant** vector store backing for high-performance semantic search. The [`src/lib/memory/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/settings.ts) file centralized configuration through environment variables:

- `MEMORY_MAX_SIZE`: Hard limit on total memory entries per scope
- `MEMORY_TYPED_DECAY_*`: Type-specific TTL configurations (factual, episodic, semantic)
- Vector store toggles for enabling cosine similarity vs. pure FTS5

The [`src/lib/memory/types.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/types.ts) module defines the `MemoryType` enum categorizing records as `FACTUAL`, `EPISODIC`, `SEMANTIC`, `PROCEDURAL`, or `SUMMARY`, allowing the retrieval engine to weight different memory categories during scoring.

## REST API Endpoints for Memory Management

OmniRoute exposes memory operations through authenticated REST endpoints:

### List memories with filtering

```bash
curl "https://api.omniroute.dev/api/memory?sessionId=sess_abc123&type=FACTUAL" \
  -H "Authorization: Bearer <management-api-key>"

```

### Create a manual memory entry

```http
POST /api/memory HTTP/1.1
Content-Type: application/json
Authorization: Bearer <management-api-key>

{
  "content": "User prefers Python over JavaScript for data tasks",
  "type": "EPISODIC",
  "sessionId": "sess_abc123",
  "apiKeyId": "key_xyz"
}

```

### Search with hybrid scoring

```bash
curl "https://api.omniroute.dev/api/memory/search?q=programming+preferences&limit=5"

```

Returns JSON containing memory IDs, content, and relevance scores generated by `getRelevanceScore()`.

### Trigger administrative summarization

```http
POST /api/memory/summarize?olderThanDays=30 HTTP/1.1
Authorization: Bearer <admin-api-key>

```

This executes `summarizeMemoriesOlderThan()`, compressing memories older than 30 days into `SUMMARY` type records.

### Disable memory for specific requests

```http
POST /api/v1/chat/completions HTTP/1.1
x-omniroute-no-memory: true
Content-Type: application/json

{
  "model": "gpt-4o",
  "messages": [{"role": "user", "content": "Execute one-off command"}]
}

```

Setting the header bypasses `retrieveMemories()`, reducing latency and token consumption for stateless queries.

## Summary

- **Extraction** captures factual statements from LLM responses in [`src/lib/memory/extraction.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/extraction.ts), storing them as `MemoryType.FACTUAL` records in SQLite immediately after streaming completes
- **Injection** inserts relevant context into prompts via [`src/lib/memory/injection.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/injection.ts), controllable through the `x-omniroute-no-memory` header or `noMemory` flag
- **Retrieval** employs hybrid FTS5 and vector search in [`src/lib/memory/retrieval.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/retrieval.ts), ranking results with `getRelevanceScore()` from [`retrieval/scoring.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/retrieval/scoring.ts)
- **Summarization** compresses historical memories using [`src/lib/memory/summarization.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/summarization.ts) to enforce limits set by `MEMORY_MAX_SIZE` and decay settings
- All stages operate through the REST API surface documented in [`docs/reference/API_REFERENCE.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/reference/API_REFERENCE.md), supporting manual CRUD operations and administrative maintenance

## Frequently Asked Questions

### How does OmniRoute decide which memories to inject into a conversation?

OmniRoute uses `shouldInjectMemory()` in [`src/lib/memory/injection.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/injection.ts) to check for opt-out flags, then calls `retrieveMemories()` which ranks candidates using hybrid FTS5 full-text search and vector similarity from [`src/lib/memory/vectorStore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/vectorStore.ts). The scoring function `getRelevanceScore()` weights results by semantic proximity to the current query and recency, returning only the top-ranked entries that fit within the configured token budget.

### Can I disable memory persistence for specific API requests?

Yes. Send the header `x-omniroute-no-memory: true` or include `"noMemory": true` in the request body to bypass the entire injection pipeline. This prevents `retrieveMemories()` from executing, ensuring the request processes with zero historical context and reduced latency.

### What is the difference between FACTUAL and SUMMARY memory types?

`FACTUAL` memories contain direct extracted statements from LLM responses, created by `extractFacts()` in [`src/lib/memory/extraction.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/extraction.ts). `SUMMARY` memories are compressed aggregations generated by `summarizeMemories()` in [`src/lib/memory/summarization.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/summarization.ts), which condense multiple older factual or episodic entries into dense narrative form to save token space while preserving conversational continuity.

### How does the summarization process prevent memory from growing indefinitely?

The `summarizeMemoriesOlderThan()` function in [`src/lib/memory/summarization.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/summarization.ts) runs periodically (or via administrative endpoint) to batch-process memories exceeding age thresholds defined by `MEMORY_TYPED_DECAY_*` environment variables. These batches are sent to the LLM with condensation prompts, and the resulting compact text replaces the original entries as new `SUMMARY` type records, enforcing the `MEMORY_MAX_SIZE` limits configured in [`src/lib/memory/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/settings.ts).