Understanding the Persistence Strategy in OpenMAIC: Hybrid Client-Server Architecture

OpenMAIC implements a hybrid persistence strategy that stores user data primarily in the browser using IndexedDB and localStorage, with optional synchronization to a PostgreSQL backend when the NEXT_PUBLIC_PERSISTENCE environment variable is enabled.

The THU-MAIC/OpenMAIC repository employs a multi-layered persistence strategy designed for offline-first responsiveness while maintaining the capability to back up data centrally. This approach ensures that workspace layouts, generated content, and user settings remain available even without network connectivity, degrading gracefully when server APIs are unreachable according to the source code.

Client-Side Storage Architecture

The foundation of OpenMAIC's persistence strategy relies on browser-based storage mechanisms that guarantee immediate data availability and atomic writes.

Document-Level Persistence with Dexie

At the core of the client-side layer, lib/pbl/v2/runtime/document-persistence.ts implements an IndexedDB schema using the Dexie wrapper. This module manages three primary object stores: documents, scenes, and metadata. The implementation exposes critical methods including saveDocument, loadDocument, and flushScenes, which handle batched transactional writes to ensure data consistency.

Changes are queued in the state store and flushed on a 500 ms debounce, writing batch transactions to Dexie to guarantee atomicity. The module also provides preparePBLScenesForDocumentPersistence, a helper function that sanitizes scene data before writing to prevent schema corruption.

Lightweight Key-Value Storage

For lightweight configuration data, OpenMAIC uses BrowserKVStore defined in lib/document-store/persistence-types.ts. This utility wraps localStorage to persist user settings such as selected models, API keys, and UI preferences. Unlike the Dexie implementation, this store persists data immediately on every set operation, ensuring that critical preferences are never lost during unexpected browser closures.

Optional Server-Side Synchronization

When centralized backup is required, OpenMAIC extends its persistence strategy to include server-side storage without compromising the offline-first design.

Activation and API Routes

Server synchronization activates only when NEXT_PUBLIC_PERSISTENCE=1 is set. In this mode, the client sends PATCH requests to /api/persistence/documents after local Dexie writes succeed. The Next.js API routes handle validation and write operations to PostgreSQL, returning the persisted document ID for reference tracking. The lib/server/folder-persistence.ts module complements this by handling workspace folder metadata through dedicated API handlers.

Fallback Behavior

The architecture prioritizes local data durability. When server synchronization fails—due to network outages or API errors—the system falls back to the local IndexedDB copy without deletion or corruption. The source code explicitly tests this behavior to verify that failed server writes do not compromise locally stored data.

Data Integrity and Conflict Resolution

OpenMAIC's persistence strategy includes sophisticated mechanisms to handle concurrent modifications and delete-restore cycles.

Epoch-Based Fencing

To prevent stale writes from overwriting newer data during delete-then-restore operations, the implementation uses an epoch counter. Each deletion increments the epoch; subsequent writes must include the current epoch value, causing the system to ignore stale writes automatically. Test suites such as "re-persists a fenced in-flight aggregate save after the deletion fails" validate this protection mechanism.

Atomic Transactions

The Dexie implementation guarantees atomicity through batch transactions. When flushScenes executes, it writes all pending changes in a single transaction, ensuring that partial writes cannot leave the document store in an inconsistent state. This transactional approach applies equally to quiz state persistence in lib/quiz/persistence.ts and workspace folder operations.

Implementation Examples

The following patterns demonstrate how to interact with OpenMAIC's persistence layers in application code.

Persisting document changes with automatic local and optional remote synchronization:

import { useStageStore } from '@/stores/stage';
import { persistDocument } from '@/lib/pbl/v2/runtime/document-persistence';

function onSlideEdit(updatedSlide) {
  useStageStore.setState({ slides: updatedSlide });
  // Persist locally (Dexie) and optionally to the server
  persistDocument(useStageStore.getState()).catch(console.error);
}

Managing user settings through the localStorage-based KV store:

import { BrowserKVStore } from '@/lib/document-store/persistence-types';

const kv = new BrowserKVStore({ storage: localStorage });
await kv.set('settings', { model: 'gpt-4', theme: 'dark' });
const saved = await kv.get<{ model: string; theme: string }>('settings');
console.log(saved);

Conditionally executing server synchronization based on environment configuration:

if (process.env.NEXT_PUBLIC_PERSISTENCE === '1') {
  fetch('/api/persistence/documents', {
    method: 'PATCH',
    body: JSON.stringify({ documentId, payload }),
    headers: { 'Content-Type': 'application/json' },
  });
}

Summary

• Hybrid Architecture: OpenMAIC combines client-side IndexedDB (via Dexie) and localStorage with optional PostgreSQL synchronization for resilient data storage.
• Atomic Operations: Document changes are batched and flushed with 500ms debouncing, ensuring transactional integrity through Dexie's batch writes.
• Conflict Prevention: Epoch-based fencing prevents stale data from overwriting fresh content during deletion and restoration cycles.
• Graceful Degradation: When NEXT_PUBLIC_PERSISTENCE is disabled or server APIs fail, the system maintains full functionality using local storage without data loss.
• Modular Implementation: Specific persistence concerns are separated into targeted modules including document-persistence.ts, folder-persistence.ts, and quiz/persistence.ts.

Frequently Asked Questions

Where does OpenMAIC store user data when working offline?

According to the OpenMAIC source code, offline data persists entirely within the browser using IndexedDB for complex document structures (via the Dexie wrapper in lib/pbl/v2/runtime/document-persistence.ts) and localStorage for lightweight key-value settings (via BrowserKVStore). This design ensures that slides, chat history, and workspace layouts remain accessible without network connectivity.

How does OpenMAIC prevent data loss when syncing to the server fails?

The persistence strategy implements a fallback mechanism that prioritizes local storage durability. When a PATCH request to /api/persistence/documents fails, the system retains the IndexedDB copy and does not delete or corrupt local data. Tests in the repository explicitly verify that failed server writes do not impact the local document state, allowing users to continue working and retry synchronization later.

What is the purpose of the epoch counter in OpenMAIC's persistence system?

The epoch counter serves as a fencing mechanism to prevent race conditions during delete-restore operations. When a document or scene is deleted, the epoch increments; subsequent writes must match the current epoch to be accepted. This ensures that in-flight save operations from previous states cannot overwrite newer data, maintaining consistency during rapid edit-delete-restore cycles as implemented in the core persistence logic.

Which environment variable controls server-side persistence in OpenMAIC?

Server-side persistence is gated by the NEXT_PUBLIC_PERSISTENCE environment variable. When set to 1, the client activates synchronization routes that write to PostgreSQL after local IndexedDB operations complete. If undefined or set to 0, the application operates in offline-only mode using browser storage exclusively, as handled by the conditional logic in the runtime persistence modules.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}