# How OmniRoute Manages Conversational Context: Extraction, Injection, Retrieval, and Summarization > Discover how OmniRoute manages conversational context with its four-stage memory pipeline: extraction, injection, retrieval, and summarization. Learn how it uses SQLite for efficient context handling. - Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute) - Tags: deep-dive - Published: 2026-07-25 --- **OmniRoute implements a persistent four-stage memory pipeline that extracts facts from LLM responses, injects relevant context into prompts, retrieves memories via hybrid FTS5 and vector search, and summarizes aging memories to maintain token efficiency—all backed by SQLite.** OmniRoute is an open-source LLM gateway that maintains conversational continuity across requests through a sophisticated memory subsystem. Unlike simple prompt buffering, this system persists facts, summaries, and semantic embeddings in SQLite, enabling long-running conversations that exceed typical context windows. The architecture operates through four distinct stages—extraction, injection, retrieval, and summarization—each implemented as dedicated modules in the `src/lib/memory/` directory. ## Stage 1: Conversational Context Extraction The extraction phase captures factual information from LLM responses for persistent storage. In [`src/lib/memory/extraction.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/extraction.ts), the pipeline defines the `ExtractedFact` interface and exposes `extractFactsFromText()` and the async `extractFacts()` functions. Afterstreaming a response back to the client, OmniRoute parses the LLM output to identify factual statements. Each extracted fact becomes a new row in the `memories` table with `MemoryType.FACTUAL`. For vector-based semantic storage, the system also creates `MemoryType.SEMANTIC` records using the minimal SQLite-vector implementation found in [`src/lib/memory/vectorStore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/vectorStore.ts). ```typescript // Conceptual flow in extraction.ts interface ExtractedFact { content: string; confidence: number; timestamp: Date; } async function extractFacts(text: string): Promise ``` This process ensures that user preferences, stated facts, and key details survive beyond the immediate request lifecycle. ## Stage 2: Memory Injection into Prompts Before sending requests upstream, OmniRoute evaluates whether to enrich the prompt with historical context. The [`src/lib/memory/injection.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/injection.ts) module contains `shouldInjectMemory()`, `injectMemory()`, and `formatMemoryContext()` to handle this logic. When enabled, `retrieveMemories()` fetches relevant records by session ID, API key, or custom query, then `formatMemoryContext()` concatenates these into a system message or appends them to the user prompt. Developers can bypass this behavior entirely by including the `x-omniroute-no-memory: true` header or setting the `noMemory` request flag, which forces `shouldInjectMemory()` to return false and saves token budget. ```http POST /api/v1/chat/completions HTTP/1.1 x-omniroute-no-memory: true Content-Type: application/json Authorization: Bearer { "model": "gpt-4o", "messages": [{ "role": "user", "content": "What is the weather?" }] } ``` This injection mechanism ensures the LLM receives pertinent background without manual conversation management. ## Stage 3: Hybrid Memory Retrieval (FTS5 + Vector Search) Retrieving the most relevant context requires balancing keyword precision with semantic similarity. The [`src/lib/memory/retrieval.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/retrieval.ts) module implements `retrieveMemories()`, `retrievePreview()`, and `engineStatus()` to orchestrate this hybrid approach. The system combines **SQLite FTS5** full-text search with optional vector similarity queries against the built-in vector store. Results are ranked using `getRelevanceScore()` from [`src/lib/memory/retrieval/scoring.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/retrieval/scoring.ts), which balances exact match accuracy with embedding proximity. This dual strategy ensures that specific terms (like product names or dates) surface alongside conceptually related content (like "vacation plans" matching "trip to Japan"). ```bash curl "https://my.omniroute.dev/api/memory/search?q=travel&limit=5" ``` The JSON response includes `id`, `content`, and a relevance score generated by the scoring engine, allowing the injection layer to prioritize the most pertinent memories. ## Stage 4: Memory Summarization for Token Efficiency As conversations grow, raw memory accumulation threatens to exceed token limits. OmniRoute addresses this through [`src/lib/memory/summarization.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/summarization.ts), which provides `summarizeMemories()` and `summarizeMemoriesOlderThan()` to compress aging records. When triggered—either periodically or via admin endpoint—the system sends batches of memories to the LLM with a specialized summarization prompt. The resulting condensed text is stored as a new memory record with `MemoryType.SUMMARY`. This compaction reduces the total token count while preserving essential context boundaries defined in [`src/lib/memory/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/settings.ts) via `MEMORY_MAX_SIZE` and `MEMORY_TYPED_DECAY_*` environment variables. ```http POST /api/memory/summarize?olderThanDays=30 HTTP/1.1 Authorization: Bearer ``` The endpoint processes eligible memories, generates summaries, and writes them back to the database, maintaining long-term conversational coherence without unbounded growth. ## Memory Types and Configuration All memory records conform to the `MemoryType` enum defined in [`src/lib/memory/types.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/types.ts): - **FACTUAL**: Explicit facts extracted from LLM outputs - **EPISODodic**: Specific conversation events or interactions - **SEMANTIC**: Vector embeddings for similarity search - **PROCEDURAL**: System-level patterns or instructions - **SUMMARY**: Compressed representations of memory batches Configuration defaults and limits reside in [`src/lib/memory/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/settings.ts), allowing operators to tune retention policies, decay rates, and maximum storage thresholds without code changes. ## REST API for Memory Operations OmniRoute exposes the memory subsystem through REST endpoints documented in [`docs/reference/API_REFERENCE.md`](https://github.com/diegosouzapw/OmniRoute/blob/main/docs/reference/API_REFERENCE.md): - **List memories**: `GET /api/memory` (filters: `apiKeyId`, `type`, `sessionId`, `q`) - **Create memory**: `POST /api/memory` (validated by `MemoryCreateInputSchema`) - **Retrieve single memory**: `GET /api/memory/[id]` - **Search**: `GET /api/memory/search` (FTS5 + vector hybrid) - **Clear memories**: `POST /api/memory/clear` - **Health check**: `GET /api/memory/health` Manual memory creation allows external systems to inject domain knowledge: ```http POST /api/memory HTTP/1.1 Content-Type: application/json Authorization: Bearer { "content": "User said they love sushi.", "type": "EPISODIC", "sessionId": "sess_abc123", "apiKeyId": "key_xyz" } ``` ## Summary - **Extraction** occurs via [`src/lib/memory/extraction.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/extraction.ts), where `extractFactsFromText()` parses LLM responses into `FACTUAL` and `SEMANTIC` memory records stored in SQLite. - **Injection** happens in [`src/lib/memory/injection.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/injection.ts) through `injectMemory()` and `formatMemoryContext()`, which can be disabled per-request using the `x-omniroute-no-memory` header. - **Retrieval** combines FTS5 and vector search in [`src/lib/memory/retrieval.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/retrieval.ts), scoring results with `getRelevanceScore()` to find relevant context. - **Summarization** compresses older memories via `summarizeMemoriesOlderThan()` in [`src/lib/memory/summarization.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/summarization.ts), creating `SUMMARY` type records to bound token usage. - The system supports five memory types defined in [`types.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/types.ts), with configurable limits in [`settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/settings.ts) and full CRUD access via REST endpoints. ## Frequently Asked Questions ### How does OmniRoute extract facts from LLM responses? After streaming a response to the client, OmniRoute runs `extractFacts()` from [`src/lib/memory/extraction.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/extraction.ts), which calls `extractFactsFromText()` to identify factual statements. These are persisted as `MemoryType.FACTUAL` records in SQLite, while semantic embeddings are stored as `MemoryType.SEMANTIC` via the vector store. ### Can I disable memory injection for specific requests? Yes. Include the header `x-omniroute-no-memory: true` or set the `noMemory` flag in the request payload. This causes `shouldInjectMemory()` in [`src/lib/memory/injection.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/injection.ts) to skip the retrieval and formatting steps, preventing context from being added to that specific prompt. ### What search methods does OmniRoute use for memory retrieval? OmniRoute employs a hybrid approach in [`src/lib/memory/retrieval.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/retrieval.ts) that combines SQLite FTS5 for full-text search with optional vector similarity search through the built-in [`vectorStore.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/vectorStore.ts). The `getRelevanceScore()` function ranks results to surface the most pertinent memories for injection. ### How does OmniRoute handle memory size limits? The system uses `summarizeMemories()` and `summarizeMemoriesOlderThan()` from [`src/lib/memory/summarization.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/summarization.ts) to compress batches of older memories into concise `SUMMARY` records. This process respects the `MEMORY_MAX_SIZE` and `MEMORY_TYPED_DECAY_*` environment variables defined in [`src/lib/memory/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/settings.ts), ensuring token usage remains bounded while preserving critical conversational context.