# How to Use the Memory System for Persistent Conversational Context in OmniRoute

> Master OmniRoute's memory system to achieve persistent conversational context. Learn how this modular pipeline maintains context across sessions using vector search.

- Repository: [Diego Rodrigues de Sa e Souza/OmniRoute](https://github.com/diegosouzapw/OmniRoute)
- Tags: how-to-guide
- Published: 2026-07-26

---

**OmniRoute provides a modular pipeline that extracts, embeds, stores, and retrieves conversation snippets using vector search to maintain context across disconnected sessions.**

OmniRoute ships with a full-featured memory subsystem that persists conversational context across multiple requests through a durable vector store implementation. By leveraging **Qdrant** or **SQLite** backends combined with intelligent chunking in [`src/lib/memory/extraction.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/extraction.ts), the system remains **stateless** from the request handler perspective while enabling horizontal scaling and coherent multi-turn interactions.

## Architectural Overview

The memory pipeline follows a strict lifecycle implemented across dedicated modules in `src/lib/memory/`. Each request flows through six phases: extraction, optional summarization, embedding, storage, retrieval, and injection.

### Extraction and Chunking

In [`src/lib/memory/extraction.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/extraction.ts), incoming message payloads are parsed and sliced into discrete chunks. Each chunk receives metadata tags including a **conversation-id** and **turn-index**, creating traceable fragments ready for vectorization.

### Vectorization and Storage

Chunks are converted to 1536-dimensional embeddings via the `vectorize()` function from the embeddings service. These vectors persist in either **Qdrant** ([`src/lib/memory/qdrant.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/qdrant.ts)) or **SQLite** ([`src/lib/memory/store.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/store.ts)), depending on the configured provider. The [`store.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/store.ts) module exposes `storeChunk()`, which orchestrates extraction, embedding, and persistence as a high-level façade.

### Retrieval and Injection

When a new request arrives, [`src/lib/memory/retrieval.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/retrieval.ts) executes a similarity search using `retrieveRelevantChunks()`. The function embeds the current user prompt and performs nearest-neighbour lookup against the vector store. Retrieved snippets are then merged into the request payload by [`src/lib/memory/injection.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/injection.ts), either prepended to the prompt or inserted as system messages.

## Configuration and Initialization

Runtime behavior is controlled through [`src/lib/memory/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/settings.ts) and validated by [`src/lib/memory/verify.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/verify.ts) using Zod schemas.

### Environment Settings

Key toggles include `MEMORY_ENABLED`, `MEMORY_MAX_TOKENS`, and the vector store provider selection. Configure these via the settings API endpoint at [`src/app/api/settings/memory/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/settings/memory/route.ts):

```http
POST /api/settings/memory
Content-Type: application/json

{
  "enabled": true,
  "maxTokens": 2048,
  "topK": 5,
  "provider": "qdrant"
}

```

This endpoint validates the payload and persists configuration to the database.

## Storing Conversation Context

To persist a turn programmatically, import the storage façade from [`src/lib/memory/store.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/store.ts):

```typescript
import { storeChunk } from "@/lib/memory/store";

// Persist a user message with session metadata
await storeChunk({
  conversationId: convId,
  messageId: msg.id,
  content: msg.content,
  metadata: { role: "user", timestamp: Date.now() },
});

```

Behind the scenes, this triggers [`extraction.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/extraction.ts) to chunk the text, calls the embedding service, and commits the vector plus metadata to your configured backend.

## Retrieving Context for New Requests

Access historical context using the retrieval module before forwarding requests to your LLM executor:

```typescript
import { retrieveRelevantChunks } from "@/lib/memory/retrieval";

const context = await retrieveRelevantChunks({
  conversationId,
  prompt: newUserPrompt,
  topK: 5,
});

const enrichedPrompt = `${context.join("\n")}\n\n${newUserPrompt}`;

```

The function returns plain text snippets ranked by semantic similarity, ready for concatenation with new user input.

## Integration in Request Pipeline

The complete lifecycle is orchestrated by [`src/app/api/memory/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/memory/route.ts) through its `handleMemory` function:

```typescript
import { handleMemory } from "@/app/api/memory/route";

export async function POST(req: Request) {
  // Execute full pipeline: extract → store → retrieve → inject
  const enrichedReq = await handleMemory(req);
  
  // Forward enriched payload to target model
  const response = await executor.execute(enrichedReq);
  return response;
}

```

This design ensures the request handler remains stateless while the durable vector store maintains continuity across disconnected sessions.

## Performance Optimization and Caching

The system implements an **LRU cache** in [`src/lib/memory/cache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/cache.ts) to prevent redundant embedding API calls. When identical chunks are encountered, cached vectors are reused, reducing latency and API quota consumption.

For background maintenance, [`src/lib/memory/reindex.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/reindex.ts) handles schema migrations and vector database synchronization without blocking live traffic.

## Edge Cases and Fault Tolerance

**Token Budget Enforcement:** [`settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/settings.ts) caps injected context at `MEMORY_MAX_TOKENS`. When retrieved chunks exceed this limit, the system truncates from the oldest entries forward.

**Privacy and Isolation:** Memory scopes can be restricted per API key. Delete historical data via the settings route using `DELETE /api/settings/memory`.

**Graceful Degradation:** If the vector store becomes unavailable, [`verify.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/verify.ts) triggers a fallback to **no-memory mode**, allowing requests to proceed without context rather than failing entirely.

## Summary

- OmniRoute's memory system uses a **six-stage pipeline** (extraction → summarization → embedding → storage → retrieval → injection) implemented in `src/lib/memory/`
- **Stateless request handlers** ensure horizontal scalability while the vector store maintains durability
- Configure via [`src/app/api/settings/memory/route.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/app/api/settings/memory/route.ts) using environment variables like `MEMORY_ENABLED` and `MEMORY_MAX_TOKENS`
- Use `storeChunk()` from [`src/lib/memory/store.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/store.ts) to persist turns and `retrieveRelevantChunks()` from [`src/lib/memory/retrieval.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/retrieval.ts) to fetch context
- **LRU caching** in [`src/lib/memory/cache.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/cache.ts) optimizes embedding costs by avoiding duplicate vector calculations

## Frequently Asked Questions

### What vector store backends does OmniRoute support?

OmniRoute ships with adapters for **Qdrant** ([`src/lib/memory/qdrant.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/qdrant.ts)) and **SQLite** ([`src/lib/memory/store.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/store.ts)). You can extend support for alternatives like Pinecone by implementing the `VectorStore` interface used by the storage façade.

### How does OmniRoute handle token limits when injecting memory?

The system enforces a `MEMORY_MAX_TOKENS` budget defined in [`src/lib/memory/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/settings.ts). Retrieved chunks are trimmed from the oldest entries when the cumulative token count exceeds this threshold, ensuring the final prompt stays within model context windows.

### Can conversations be isolated between different API keys?

Yes. The memory system supports per-key scoping configured through [`src/lib/memory/settings.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/settings.ts). You can purge data for specific keys via `DELETE` requests to `/api/settings/memory`, making it suitable for multi-tenant deployments.

### What happens if the vector database is unavailable?

According to [`src/lib/memory/verify.ts`](https://github.com/diegosouzapw/OmniRoute/blob/main/src/lib/memory/verify.ts), the pipeline implements graceful degradation. If the vector store connection fails, the system automatically falls back to **no-memory mode**, processing requests without historical context rather than returning errors.