How the Lemon AI Experience Repository Stores and Retrieves User Interaction Memories

The Lemon AI experience repository persists user interaction memories as JSON files through the FileStorage class, implementing the MemoryStorage abstract interface to provide CRUD operations and full-text search across categorized knowledge caches.

The hexdocom/lemonai project treats experience data as generic memories that can be persisted, queried, and updated by the experience repository. This system enables the AI to maintain a searchable history of user-agent interactions that can be consulted during planning, coding, or reflection phases.

Architecture of the Experience Repository

The repository is built on a three-layer abstraction that separates storage concerns from business logic.

The MemoryStorage Abstract Interface

At the core of the system, src/knowledge/MemoryStorage.js defines the contract that all storage implementations must follow. This abstract API specifies six required methods: save, get, getAll, update, delete, and search. Any concrete storage provider must implement these methods to be compatible with the experience repository.

FileStorage Implementation Details

The primary implementation resides in src/knowledge/FileStorage.js, which persists each memory as a JSON file under a knowledge cache directory. This class handles:

• ID generation: UUID v4 for new memories without IDs
• Metadata management: Automatic timestamp (defaulting to Date.now()) and importance (defaulting to 1)
• Directory structure: Files stored under Caches/knowledge/<category>/<id>.json
• Serialization: Pretty-printed JSON output for human readability

Database Fallback via Knowledge Model

When relational database storage is preferred, the system falls back to the Knowledge Sequelize model. The utility functions in src/knowledge/knowledge.util.js act as a bridge, switching between FileStorage and the database implementation based on runtime context and configuration.

Storage and Retrieval Workflow

The experience repository follows a consistent pattern for all memory operations, accessible through the resolveStorage utility.

Initializing Storage with resolveStorage

Before performing any operations, the system creates a storage instance tied to a specific category. The default categories are "planning" and "coding".

const { resolveStorage } = require('@src/knowledge/knowledge.util');

// Initialize storage for the planning category
const storage = resolveStorage('planning');

The resolveStorage function constructs a FileStorage instance mapped to the appropriate subdirectory under Caches/knowledge.

Saving Memories with Auto-Generated Metadata

When creating new user interaction memories, the save method (lines 37-45 of FileStorage.js) automatically populates missing fields:

1. Generates a UUID v4 if id is absent
2. Sets timestamp to Date.now() if missing
3. Assigns importance to 1 if unspecified
4. Writes the object as pretty-printed JSON to the category folder

async function addExperience(content) {
  const storage = resolveStorage('planning');
  const memory = { content, importance: 2 };
  const saved = await storage.save(memory);
  console.log('Saved memory ID:', saved.id);
}

Retrieving Individual Memories

The get method (lines 47-55 of FileStorage.js) resolves the file path from the ID, reads the JSON file, and parses it into a memory object. If the file does not exist, it returns null rather than throwing an error.

async function getExperience(id) {
  const storage = resolveStorage('planning');
  const memory = await storage.get(id);
  return memory; // Returns null if not found
}

Listing All Memories in a Category

To retrieve the complete history of interactions, the getAll method (lines 58-75 of FileStorage.js) reads every .json file in the category directory, parses each one, and returns them as an array.

async function listExperiences() {
  const storage = resolveStorage('coding');
  const all = await storage.getAll();
  console.log(`Found ${all.length} memories`);
  return all;
}

Updating and Deleting Memories

Updating a memory (lines 77-83 of FileStorage.js) retrieves the current object, merges the supplied updates, adds an updatedAt timestamp, and re-saves the file.

Deleting a memory (lines 85-93 of FileStorage.js) removes the JSON file and returns true on success or false if the file was already absent.

async function updateExperience(id, newContent) {
  const storage = resolveStorage('planning');
  const updated = await storage.update(id, { content: newContent });
  return updated;
}

async function deleteExperience(id) {
  const storage = resolveStorage('planning');
  const success = await storage.delete(id);
  return success;
}

Full-Text Search Capabilities

The search method (lines 96-107 of FileStorage.js) enables querying the experience repository without exact ID matches. It loads all memories in the category, performs case-insensitive substring matching against the content field and any metadata, then sorts results by newest timestamp and applies limit/offset pagination.

async function searchExperiences(query) {
  const storage = resolveStorage('planning');
  const results = await storage.search(query, { limit: 20, offset: 0 });
  return results;
}

Front-End Integration

The front-end service at frontend/src/services/experience.js provides the bridge between the user interface and the back-end storage system. This service calls the /api/experience/* routes, which eventually invoke the storage utilities described above.

This architecture allows the UI to:

• Create new experiences via storage.save
• Read individual experiences via storage.get
• List all experiences via storage.getAll
• Search experiences via storage.search
• Update or Delete experiences via storage.update and storage.delete

The public/schemas/experience.json file defines the JSON schema that validates the shape of experience objects exposed through the API, ensuring type safety between the front-end and the repository layer.

Summary

• The experience repository in Lemon AI uses the MemoryStorage abstract interface to define storage contracts in src/knowledge/MemoryStorage.js.
• FileStorage in src/knowledge/FileStorage.js provides the default implementation, writing memories as JSON files to Caches/knowledge/<category>/.
• The resolveStorage utility in src/knowledge/knowledge.util.js initializes storage instances and can switch to the Knowledge DB model when configured.
• All memories receive auto-generated UUID v4 IDs, timestamps, and importance scores upon creation.
• The repository supports full CRUD operations plus full-text search with pagination, sorting results by recency.
• Front-end integration occurs through frontend/src/services/experience.js, which communicates with back-end routes to manipulate the underlying storage.

Frequently Asked Questions

What file format does the Lemon AI experience repository use to store memories?

The experience repository stores each memory as an individual JSON file. The FileStorage class writes pretty-printed JSON objects to the file system under Caches/knowledge/<category>/<id>.json, making the data human-readable and portable while maintaining structure through the JSON schema defined in public/schemas/experience.json.

How does the experience repository handle missing fields when saving a new memory?

According to the source code in src/knowledge/FileStorage.js (lines 37-45), the repository automatically populates three critical fields if absent: it generates a UUID v4 for the id, sets the timestamp to the current time using Date.now(), and assigns an importance value of 1. This ensures every memory has complete metadata for sorting and retrieval.

Can the experience repository switch from file-based storage to a database?

Yes. The knowledge.util.js file contains logic that can instantiate either FileStorage or the Knowledge Sequelize model depending on configuration. While FileStorage is the default for local caching, the repository abstraction allows seamless substitution with relational database storage when persistence requirements demand it.

How does the search functionality work in the experience repository?

The search method in FileStorage.js (lines 96-107) performs a case-insensitive substring match across the content field and any metadata properties of stored memories. It loads all memories in the requested category, filters for matches, sorts them by newest timestamp first, then applies limit and offset parameters to return paginated results suitable for UI display.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}