How Does @openmaic/storage Handle Data Persistence?

The @openmaic/storage package implements a scope-aware key-value abstraction that separates account (cloud-synced) and device (local-only) data, backed by pluggable backends including localStorage, HTTP, and PostgreSQL, alongside a content-addressed asset layer for binary blobs.

The @openmaic/storage package serves as the official persistence layer for the THU-MAIC/OpenMAIC framework. It decouples storage semantics from storage mechanics, enabling developers to swap between browser, server, and database backends without altering application business logic while enforcing strict data residency guarantees.

Core Architecture: The KVStore Primitive and Scope Safety

The foundation of @openmaic/storage is the KVStore interface defined in src/kv/types.ts. This contract specifies four essential operations—get, set, remove, and keys—each accepting an optional KVScope parameter that determines data residency behavior.

The package recognizes two distinct scopes:

• account – Data syncs across all user devices via remote backends
• device – Data remains strictly local to the originating machine

To prevent accidental data leakage, the source code implements DeviceSafeKVStore, a branded interface that exposes a read-only servesDeviceScopeLocally: true flag. As implemented in src/kv/types.ts (lines 40-42), this flag ensures TypeScript can validate at compile time that a remote-only store (like HttpKVStore) never handles device-scoped data. Runtime enforcement occurs via assertKVScope, which throws KVScopeViolationError for invalid scope literals (lines 83-92).

Browser Storage: Implementing Device-Safe Persistence with localStorage

For client-side persistence, BrowserKVStore wraps the Web Storage API. Located in src/kv/browser.ts, this backend serializes values as JSON strings and namespaces keys using the format maic:{scope}:{key} (lines 37-41).

The implementation validates data before writing: functions, symbols, and objects with toJSON methods returning undefined are explicitly rejected (lines 69-81). This ensures compatibility with the HTTP backend and prevents serialization errors. When initialized with scope 'device', BrowserKVStore automatically satisfies the DeviceSafeKVStore contract, guaranteeing that sensitive local data never enters sync pipelines.

Remote Sync: HTTP Backend for Account-Scoped Data

The HttpKVStore class (implemented in src/kv/http.ts) provides server-side persistence for account-scoped data. It communicates with RESTful endpoints using standard HTTP verbs while handling serialization and error recovery.

Crucially, this backend enforces scope restrictions at the network boundary. Attempting to write device-scoped data through HttpKVStore results in runtime rejection, aligning with the package's security model that mandates local-only storage for sensitive device information.

React State Persistence: Zustand Integration

To bridge React state management with the storage layer, the package exports kvPersistStorage, a Zustand middleware adapter defined in src/zustand/persist.ts. This utility transforms any KVStore implementation into the storage shape expected by Zustand's persist middleware.

The adapter exposes getItem, setItem, and removeItem methods that forward to the underlying store using the specified scope (lines 59-75). TypeScript overloads provide compile-time safety: passing a non-device-safe store with scope 'device' generates a type error, preventing developers from accidentally syncing local encryption keys or session tokens to remote servers.

import create from 'zustand';
import { persist } from 'zustand/middleware';
import { BrowserKVStore, kvPersistStorage } from '@openmaic/storage';

const kv = new BrowserKVStore({ namespace: 'myapp' });

export const useSettings = create(
  persist(
    (set) => ({
      theme: 'light',
      setTheme: (t: string) => set({ theme: t }),
    }),
    {
      name: 'settings-storage',
      storage: kvPersistStorage(kv, 'device'),
    },
  ),
);

Binary Asset Management: IndexedDB and Content Hashing

Beyond key-value pairs, @openmaic/storage manages large binary objects through BrowserAssetStore in src/asset/browser-store.ts. This class provides a content-addressed, reference-counted blob pool backed by IndexedDB.

The architecture separates metadata from bytes:

• Registry store – Maps AssetId to {contentHash, mime, meta} records
• Blobs store – Maps contentHash to raw bytes, enabling automatic deduplication

All operations execute within single IndexedDB transactions (lines 36-70), ensuring atomic reference-count updates. The store mints stable blob: URLs cached per AssetId (lines 46-56), allowing safe usage in DOM elements without memory leaks. Calling release revokes URLs without affecting the registry, enabling precise memory management for generated media.

import { BrowserAssetStore } from '@openmaic/storage';
import type { BinaryBlob } from '@openmaic/dsl';

const assetStore = new BrowserAssetStore();
const blob: BinaryBlob = new Blob([canvas.toBlob()], { type: 'image/png' });
const assetId = await assetStore.put(blob, { 
  contentType: 'image/png', 
  filename: 'screenshot.png' 
});
const url = await assetStore.resolve(assetId);
if (url) imgElement.src = url;

Higher-Level Abstractions: Documents and Runtime State

Building upon the KV and asset primitives, the package provides typed stores for complex domain objects. DocumentStore, RuntimeStore, and AgentSessionStore expose high-level APIs for persisting complete MAIC documents, runtime sessions, and agent state.

Available implementations include:

• BrowserDocumentStore – Local document persistence using IndexedDB
• HttpDocumentStore – Cloud-synced document storage
• PgDocumentStore – PostgreSQL-backed persistence for server deployments
• BrowserRuntimeStore – Client-side session state management

These classes are re-exported from src/index.ts, providing a unified entry point for consumers regardless of their chosen backend.

Summary

• @openmaic/storage separates storage contracts from implementations through the KVStore interface and DeviceSafeKVStore branding.
• Scope enforcement prevents device-local data from leaking to remote servers via compile-time types and runtime assertions in src/kv/types.ts.
• BrowserKVStore provides localStorage-backed persistence with JSON validation and automatic namespacing.
• HttpKVStore enables account synchronization while rejecting device-scoped writes at the network boundary.
• kvPersistStorage offers type-safe Zustand integration for React applications, enforcing scope constraints through TypeScript overloads.
• BrowserAssetStore delivers content-addressed binary storage with reference counting, transaction safety, and URL lifecycle management via IndexedDB.
• Pluggable backends allow swapping between browser, HTTP, PostgreSQL, and S3 implementations without modifying business logic.

Frequently Asked Questions

What is the difference between account and device scope in @openmaic/storage?

The account scope designates data intended for cross-device synchronization, such as user preferences or cloud documents, while the device scope restricts data to the local machine for security-sensitive information like encryption keys or session tokens. As defined in src/kv/types.ts, the DeviceSafeKVStore interface ensures that only backends with servesDeviceScopeLocally: true (such as BrowserKVStore) can handle device-scoped operations.

How does @openmaic/storage prevent sensitive data from syncing to the cloud?

The package implements a two-layer defense: TypeScript branding via DeviceSafeKVStore prevents compilation of code attempting to pass non-local stores for device scope, while runtime assertions in assertKVScope throw KVScopeViolationError if scope literals are misused. Additionally, HttpKVStore explicitly rejects device-scoped writes, ensuring that local data never traverses network boundaries.

Can I use @openmaic/storage with state management libraries other than Zustand?

While the package provides a first-party adapter via kvPersistStorage in src/zustand/persist.ts, the underlying KVStore interface is framework-agnostic. Any library supporting asynchronous storage adapters (such as Redux Persist or Pinia) can consume BrowserKVStore, HttpKVStore, or PgDocumentStore directly, provided the adapter maps the store's get, set, and remove methods to the library's expected contract.

How does BrowserAssetStore handle duplicate binary data?

BrowserAssetStore calculates content hashes for all incoming blobs and stores bytes in a dedicated blobs object store indexed by hash. When storing duplicate content, the registry creates a new AssetId pointing to the existing hash without writing redundant data. Reference counting within IndexedDB transactions ensures that shared blob entries are only deleted when all referencing assets are removed, optimizing storage for duplicate images or documents.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}