How to Build a User Memory System That Persists Across Sessions: A Modular Guide

A modular, pluggable user-memory system stores facts as JSON files per user, enabling automatic loading at session start and durability through abstract manager classes that handle deduplication and conflict resolution.

The bojieli/ai-agent-book repository implements a production-ready approach to build a user memory system that persists across sessions without requiring external databases. By leveraging a three-layer architecture of configuration, memory managers, and CLI tooling, the system enables LLM agents to recall user preferences and facts between conversations using local JSON storage.

Architecture Overview

The persistent memory subsystem is organized into three distinct layers, each responsible for specific aspects of data management:

Layer	Responsibility	Main Files
Configuration	Centralizes settings, paths, and memory mode selection	chapter3/user-memory/config.py
Memory Managers	Implements storage back-ends with a common abstract API	chapter3/user-memory/memory_manager.py
CLI / Processor	Provides offline interface for testing and background processing	chapter3/user-memory/memory_cli.py

This separation allows the memory system to remain storage-agnostic while providing concrete implementations for different use cases.

Configuration and Storage Modes

The Config class in chapter3/user-memory/config.py centralizes all persistence settings. Storage locations default to data/memories and data/conversations but can be overridden via environment variables (MEMORY_STORAGE_DIR, CONVERSATION_HISTORY_DIR).

The system supports three memory modes selected via the MEMORY_MODE configuration:

• notes or enhanced_notes → Instantiates NotesMemoryManager, storing facts as simple MemoryNote objects with IDs, content, tags, and timestamps.
• json_cards → Uses JSONMemoryManager, organizing data hierarchically as category → subcategory → key → value.
• advanced_json_cards → Employs AdvancedJSONMemoryManager for complex card objects containing backstory, relationships, and metadata blocks.

All modes validate required API keys during initialization if online extraction steps are configured.

Implementing Memory Managers

All concrete managers inherit from the abstract BaseMemoryManager class and implement a standardized API: load_memory, save_memory, add_memory, update_memory, delete_memory, get_context_string, and search_memories.

Persistence is achieved by reading and writing JSON files named <user_id>_memory.json under the configured MEMORY_STORAGE_DIR:

• NotesMemoryManager stores each fact as a MemoryNote and enforces MAX_MEMORY_ITEMS by discarding oldest entries. It provides consolidate_memories() for deduplication and conflict resolution based on the first tag (used as an attribute key).

• JSONMemoryManager enables fast retrieval via search_memories() that matches on any field within the hierarchical card structure.

• AdvancedJSONMemoryManager maintains rich card objects with internal _metadata blocks while preserving the same search and consolidation semantics.

Session Persistence Mechanism

When a new conversation begins, the system creates a manager using the factory function:

from chapter3.user-memory.memory_manager import create_memory_manager

manager = create_memory_manager(user_id="alice", mode=Config.MEMORY_MODE)

The manager automatically loads the existing JSON file for that user, making every previously saved note or card immediately available. After each mutation (add_memory, update_memory, or delete_memory), the manager saves the updated structure back to disk, guaranteeing durability across sessions.

To inject memories into an LLM prompt, call get_context_string(), which formats the stored facts for context window insertion.

Consolidation, Deduplication, and Conflict Resolution

The consolidate_memories() method ensures data consistency over time through a two-step process:

1. Deduplication – Normalizes text using _normalize_text() and merges duplicates by unioning tags while keeping the newest timestamp.

2. Conflict Resolution – Groups notes by their first tag (the attribute key). When multiple notes describe the same attribute with differing content, the most recent version wins and older versions are marked as superseded.

This offline process can be run periodically via the CLI to maintain a single source of truth for each user's persistent memory.

Testing with the Command-Line Interface

The memory_cli.py tool demonstrates the full memory lifecycle without requiring LLM calls:

python chapter3/user-memory/memory_cli.py add --content "Prefers window seats" --tags "travel,seat"
python chapter3/user-memory/memory_cli.py query --query "seat"
python chapter3/user-memory/memory_cli.py consolidate
python chapter3/user-memory/memory_cli.py show

Key commands include:

• add – Insert a new note into the current session.
• query – Search across all stored sessions using search_memories().
• update – Modify an existing entry by ID.
• consolidate – Run deduplication and conflict resolution offline.
• show – Display the context string that would be fed to the LLM.
• demo – Execute an end-to-end multi-session scenario showing persistence.

All commands respect --store-path to redirect storage for safe experimentation.

Practical Implementation Examples

Initializing a Memory Manager

from chapter3.user_memory.config import Config, MemoryMode
from chapter3.user_memory.memory_manager import create_memory_manager

# Ensure storage directories exist

Config.create_directories()

# Create manager for user "alice" using default mode from .env

manager = create_memory_manager(user_id="alice")
print(manager.get_context_string())

Adding and Searching Facts

# Add a fact during the current session

note_id = manager.add_memory(
    content="Alice prefers window seats",
    session_id="session_2024_03",
    tags=["seat_preference"]
)

# Search across all sessions

results = manager.search_memories(query="seat")
for note in results:
    print(f"- {note.note_id[:8]}: {note.content}")

Running Consolidation

report = manager.consolidate_memories(resolve_conflicts=True)
print(f"Before: {report['initial_count']}, After: {report['final_count']}")
print(f"Duplicates removed: {report['duplicates_removed']}")

Extending the System

To add a new storage format to the persistent memory system:

1. Create a subclass of BaseMemoryManager implementing all abstract methods (load_memory, save_memory, etc.).
2. Add a new enum value to MemoryMode in config.py.
3. Update the factory function create_memory_manager to return the new class when the corresponding mode is selected.

Because the CLI and background processor interact only through the abstract interface, new managers integrate seamlessly without modifying downstream code.

Summary

• Modular architecture separates configuration, storage logic, and interface concerns across config.py, memory_manager.py, and memory_cli.py.
• JSON file persistence enables session-spanning storage without external databases, using <user_id>_memory.json files.
• Three storage modes (notes, json_cards, advanced_json_cards) cater to different complexity requirements while sharing a common API.
• Automatic durability occurs through the factory pattern's load/save lifecycle, ensuring data survives between conversations.
• Consolidation logic handles deduplication and conflict resolution offline, maintaining data quality over time.

Frequently Asked Questions

How does the system handle storage without a database?

The system persists user memories as individual JSON files on disk, named <user_id>_memory.json and stored in the configurable MEMORY_STORAGE_DIR. Each manager class handles serialization and deserialization automatically, loading the entire memory state at initialization and atomically saving after every mutation.

What happens when memory limits are reached?

The NotesMemoryManager enforces MAX_MEMORY_ITEMS by automatically discarding the oldest entries when the threshold is exceeded. For the JSON-based managers, no hard limit is imposed by default, though the consolidate_memories() method should be run periodically to remove duplicates and compress the stored data.

How does conflict resolution work when facts change?

During consolidation, the system groups notes by their first tag (treated as an attribute key). If multiple notes describe the same attribute with different values, the note with the most recent timestamp wins, and older versions are marked as superseded. This ensures the LLM always receives the latest user information.

Can this system integrate with existing agent frameworks?

Yes. The abstract BaseMemoryManager interface allows the memory system to plug into any LLM-driven agent. By calling get_context_string(), agents can inject formatted memory context into their prompts, while the create_memory_manager factory ensures the correct storage backend is instantiated based on environment configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}