# How Nemori's Architecture Handles Long-Term Memory for LLM Workflows: Episodes, Vectors, and Search

> Discover how Nemori's architecture manages long-term memory for LLM workflows with episode persistence vector embeddings and hybrid search for efficient context retrieval.

- Repository: [Nemori AI/nemori](https://github.com/nemori-ai/nemori)
- Tags: architecture
- Published: 2026-03-08

---

**Nemori's architecture handles long-term memory for LLM workflows by persisting conversation episodes in a relational database while asynchronously generating vector embeddings for semantic search, enabling agents to retrieve relevant historical context through hybrid vector and keyword indexing.**

Nemori (nemori-ai/nemori) implements a robust persistence layer that allows LLM-driven agents to maintain context across unlimited sessions. The system combines chronological episode storage with semantic vector indexing to create a searchable, persistent knowledge base that survives server restarts and scales from local deployments to cloud infrastructure.

## The Dual-Store Data Model

Nemori's memory system centers on two complementary structures managed by the **`MemorySystem`** class ([`src/core/memory_system.py`](https://github.com/nemori-ai/nemori/blob/main/src/core/memory_system.py)): **Episodes** and **Semantic Memories**.

### Episodes: Chronological Conversation Chunks

The **`Episode`** model ([`src/models/episode.py`](https://github.com/nemori-ai/nemori/blob/main/src/models/episode.py)) represents discrete chunks of conversation history containing messages, timestamps, and user identifiers. These records persist in a relational database through **`EpisodeStorage`** ([`src/storage/episode_storage.py`](https://github.com/nemori-ai/nemori/blob/main/src/storage/episode_storage.py)), providing durable, ACID-compliant storage for conversation history.

### Semantic Memories: Vector Encodings for Similarity Search

The **`SemanticMemory`** model ([`src/models/semantic.py`](https://github.com/nemori-ai/nemori/blob/main/src/models/semantic.py)) stores vector-encoded representations of episodes or message subsets. Generated asynchronously via the **`EmbeddingClient`** ([`src/utils/embedding_client.py`](https://github.com/nemori-ai/nemori/blob/main/src/utils/embedding_client.py)), these vectors enable cosine-similarity search and reside in a vector database (e.g., Chroma) managed by **`SemanticStorage`** ([`src/storage/semantic_storage.py`](https://github.com/nemori-ai/nemori/blob/main/src/storage/semantic_storage.py)).

## Buffering and Episode Creation

Incoming messages first accumulate in a **message buffer** ([`src/core/message_buffer.py`](https://github.com/nemori-ai/nemori/blob/main/src/core/message_buffer.py)). When the buffer reaches the configurable `BUFFER_MAX_MESSAGES` threshold or a timeout expires, Nemori triggers **`_create_episode_from_buffer`** to persist the buffered content as a new `Episode`. Immediately following creation, the system schedules asynchronous semantic generation via **`_schedule_semantic_generation`**, which computes embeddings without blocking the main workflow.

## Loading and Indexing Strategies

When initializing a user session, **`MemorySystem.load_user_data_and_indices`** executes a three-phase initialization:

1. **Fetch episodes** from `EpisodeStorage` for the specific user.
2. **Load semantic vectors** from `SemanticStorage` into memory.
3. **Build hybrid indices**:
   - **Vector index** (FAISS/Chroma via [`src/search/chroma_search.py`](https://github.com/nemori-ai/nemori/blob/main/src/search/chroma_search.py)) for semantic similarity.
   - **BM25 index** (via [`src/search/bm25_search.py`](https://github.com/nemori-ai/nemori/blob/main/src/search/bm25_search.py)) for keyword-based retrieval.

These indices cache per-user and support forced rebuilding through **`rebuild_user_indices`** when data staleness requires a refresh.

## Search and Retrieval Implementation

The public **`search`** method abstracts retrieval complexity, accepting parameters for `query`, `top_k` results, and `search_method` (`"vector"` or `"bm25"`). Internally, **`_search_episodes_by_method`** routes queries:

- **Vector search** embeds the query using `EmbeddingClient`, performs cosine-similarity lookup against the semantic index, and filters duplicates via **`_is_duplicate_semantic_memory`**.

- **BM25 search** tokenizes the query and executes inverted-index lookups through the BM25 implementation for exact keyword matching.

Both methods return `Episode` objects enriched with matched `SemanticMemory` snippets, allowing downstream LLM prompts to inject relevant historical context.

## Lifecycle Management and Consistency

### Asynchronous Processing

Semantic generation runs as background tasks managed by **`TaskManager`** ([`src/services/task_manager.py`](https://github.com/nemori-ai/nemori/blob/main/src/services/task_manager.py)). Status monitoring is available through **`get_semantic_generation_status`** and **`wait_for_semantic_generation`**, ensuring agents can verify embedding completion before dependent operations.

### Deletion and Cleanup

The **`delete_episode`** method removes episodes with optional cascading deletion of semantic vectors (`cascade_semantic=True`). For complete user data removal, **`delete_user_data`** wipes both relational and vector stores atomically.

### Thread Safety

All memory operations acquire a **per-user re-entrant lock** (**`_get_user_processing_lock`**), guaranteeing consistency when concurrent agents manipulate the same user's memory store.

## Configuration Tuning

Memory behavior is controlled via [`src/config.py`](https://github.com/nemori-ai/nemori/blob/main/src/config.py) settings including:

- `BUFFER_MAX_MESSAGES`: Trigger threshold for episode creation.
- `VECTOR_SEARCH_TOP_K`: Default semantic result count.
- `FORCE_REBUILD_INDICES`: Flag to mandate full re-indexing on next load.

These parameters adapt the subsystem for lightweight local runs or high-throughput cloud deployments.

## Code Examples

### Persisting New Conversations

```python
from nemori import MemorySystem

mem = MemorySystem()
user_id = "user-123"

# Buffer messages for automatic episode creation

mem.add_messages(
    owner_id=user_id,
    messages=[
        {"role": "user", "content": "Explain quantum computing"},
        {"role": "assistant", "content": "Quantum computing uses qubits..."}
    ]
)

# Force immediate persistence if needed

mem.force_episode_creation(owner_id=user_id)

```

### Retrieving Context with Vector Search

```python
from nemori import MemorySystem

mem = MemorySystem()
user_id = "user-123"

# Search historical episodes semantically

results = mem.search(
    owner_id=user_id,
    query="quantum entanglement applications",
    top_k=3,
    search_method="vector"
)

for episode in results:
    print(f"Episode {episode.id}: {episode.semantic_memories[0].text}")

```

### Complete User Data Removal

```python
from nemori import MemorySystem

mem = MemorySystem()
mem.delete_user_data(owner_id="user-123")

```

## Summary

- Nemori implements a **dual-store architecture** combining relational episode storage (`EpisodeStorage`) with vector semantic memories (`SemanticStorage`) for comprehensive long-term retention.
- The **`MemorySystem`** class orchestrates **asynchronous embedding generation** and **hybrid indexing** (vector + BM25) to support both similarity and keyword retrieval.
- **Per-user locking** and **configurable buffering** ensure thread-safe, high-performance operation across concurrent LLM workflows.
- All persistence mechanisms are abstracted behind simple APIs like `add_messages`, `search`, and `delete_user_data`, enabling rapid integration into agent architectures.

## Frequently Asked Questions

### How does Nemori decide when to create a new episode from buffered messages?

Nemori triggers episode creation when the message buffer reaches the `BUFFER_MAX_MESSAGES` threshold configured in [`src/config.py`](https://github.com/nemori-ai/nemori/blob/main/src/config.py) or when a timeout expires. The **`_create_episode_from_buffer`** method then persists the buffered content and schedules asynchronous semantic vector generation.

### What embedding models does Nemori use for semantic memory?

Nemori supports configurable embedding models through the **`EmbeddingClient`** utility ([`src/utils/embedding_client.py`](https://github.com/nemori-ai/nemori/blob/main/src/utils/embedding_client.py)). The specific model is deployment-dependent and generates vectors stored in the vector database for cosine-similarity searches.

### Can Nemori retrieve memories using both semantic similarity and exact keywords?

Yes. The **`MemorySystem.search`** method supports dual retrieval strategies: `"vector"` for semantic similarity search using FAISS/Chroma indices, and `"bm25"` for keyword-based retrieval using the BM25 inverted index implementation in [`src/search/bm25_search.py`](https://github.com/nemori-ai/nemori/blob/main/src/search/bm25_search.py).

### How does Nemori handle concurrent access to a single user's memory?

All memory operations acquire a per-user re-entrant lock via **`_get_user_processing_lock`** in `MemorySystem`. This mechanism guarantees thread safety when multiple concurrent agents or workflows access and modify the same user's episodic and semantic data.