# How Does @openmaic/storage Handle Data Persistence?

> Discover how @openmaic/storage handles data persistence with its scope-aware key-value abstraction, supporting localStorage, HTTP, and PostgreSQL backends for account and device data.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: internals
- Published: 2026-09-01

---

**The @openmaic/storage package implements a scope-aware key-value abstraction that separates `account` (cloud-synced) and `device` (local-only) data, backed by pluggable backends including localStorage, HTTP, and PostgreSQL, alongside a content-addressed asset layer for binary blobs.**

The `@openmaic/storage` package serves as the official persistence layer for the THU-MAIC/OpenMAIC framework. It decouples storage semantics from storage mechanics, enabling developers to swap between browser, server, and database backends without altering application business logic while enforcing strict data residency guarantees.

## Core Architecture: The KVStore Primitive and Scope Safety

The foundation of `@openmaic/storage` is the **`KVStore`** interface defined in [`src/kv/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/kv/types.ts). This contract specifies four essential operations—`get`, `set`, `remove`, and `keys`—each accepting an optional `KVScope` parameter that determines data residency behavior.

The package recognizes two distinct scopes:

- **`account`** – Data syncs across all user devices via remote backends
- **`device`** – Data remains strictly local to the originating machine

To prevent accidental data leakage, the source code implements **`DeviceSafeKVStore`**, a branded interface that exposes a read-only `servesDeviceScopeLocally: true` flag. As implemented in [`src/kv/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/kv/types.ts) (lines 40-42), this flag ensures TypeScript can validate at compile time that a remote-only store (like `HttpKVStore`) never handles `device`-scoped data. Runtime enforcement occurs via `assertKVScope`, which throws `KVScopeViolationError` for invalid scope literals (lines 83-92).

## Browser Storage: Implementing Device-Safe Persistence with localStorage

For client-side persistence, **`BrowserKVStore`** wraps the Web Storage API. Located in [`src/kv/browser.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/kv/browser.ts), this backend serializes values as JSON strings and namespaces keys using the format `maic:{scope}:{key}` (lines 37-41).

The implementation validates data before writing: functions, symbols, and objects with `toJSON` methods returning `undefined` are explicitly rejected (lines 69-81). This ensures compatibility with the HTTP backend and prevents serialization errors. When initialized with scope `'device'`, `BrowserKVStore` automatically satisfies the `DeviceSafeKVStore` contract, guaranteeing that sensitive local data never enters sync pipelines.

## Remote Sync: HTTP Backend for Account-Scoped Data

The **`HttpKVStore`** class (implemented in [`src/kv/http.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/kv/http.ts)) provides server-side persistence for `account`-scoped data. It communicates with RESTful endpoints using standard HTTP verbs while handling serialization and error recovery.

Crucially, this backend enforces scope restrictions at the network boundary. Attempting to write `device`-scoped data through `HttpKVStore` results in runtime rejection, aligning with the package's security model that mandates local-only storage for sensitive device information.

## React State Persistence: Zustand Integration

To bridge React state management with the storage layer, the package exports **`kvPersistStorage`**, a Zustand middleware adapter defined in [`src/zustand/persist.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/zustand/persist.ts). This utility transforms any `KVStore` implementation into the storage shape expected by Zustand's `persist` middleware.

The adapter exposes `getItem`, `setItem`, and `removeItem` methods that forward to the underlying store using the specified scope (lines 59-75). TypeScript overloads provide compile-time safety: passing a non-device-safe store with scope `'device'` generates a type error, preventing developers from accidentally syncing local encryption keys or session tokens to remote servers.

```typescript
import create from 'zustand';
import { persist } from 'zustand/middleware';
import { BrowserKVStore, kvPersistStorage } from '@openmaic/storage';

const kv = new BrowserKVStore({ namespace: 'myapp' });

export const useSettings = create(
  persist(
    (set) => ({
      theme: 'light',
      setTheme: (t: string) => set({ theme: t }),
    }),
    {
      name: 'settings-storage',
      storage: kvPersistStorage(kv, 'device'),
    },
  ),
);

```

## Binary Asset Management: IndexedDB and Content Hashing

Beyond key-value pairs, `@openmaic/storage` manages large binary objects through **`BrowserAssetStore`** in [`src/asset/browser-store.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/asset/browser-store.ts). This class provides a content-addressed, reference-counted blob pool backed by IndexedDB.

The architecture separates metadata from bytes:

- **Registry store** – Maps `AssetId` to `{contentHash, mime, meta}` records
- **Blobs store** – Maps `contentHash` to raw bytes, enabling automatic deduplication

All operations execute within single IndexedDB transactions (lines 36-70), ensuring atomic reference-count updates. The store mints stable `blob:` URLs cached per `AssetId` (lines 46-56), allowing safe usage in DOM elements without memory leaks. Calling `release` revokes URLs without affecting the registry, enabling precise memory management for generated media.

```typescript
import { BrowserAssetStore } from '@openmaic/storage';
import type { BinaryBlob } from '@openmaic/dsl';

const assetStore = new BrowserAssetStore();
const blob: BinaryBlob = new Blob([canvas.toBlob()], { type: 'image/png' });
const assetId = await assetStore.put(blob, { 
  contentType: 'image/png', 
  filename: 'screenshot.png' 
});
const url = await assetStore.resolve(assetId);
if (url) imgElement.src = url;

```

## Higher-Level Abstractions: Documents and Runtime State

Building upon the KV and asset primitives, the package provides typed stores for complex domain objects. **`DocumentStore`**, **`RuntimeStore`**, and **`AgentSessionStore`** expose high-level APIs for persisting complete MAIC documents, runtime sessions, and agent state.

Available implementations include:

- **`BrowserDocumentStore`** – Local document persistence using IndexedDB
- **`HttpDocumentStore`** – Cloud-synced document storage
- **`PgDocumentStore`** – PostgreSQL-backed persistence for server deployments
- **`BrowserRuntimeStore`** – Client-side session state management

These classes are re-exported from [`src/index.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/index.ts), providing a unified entry point for consumers regardless of their chosen backend.

## Summary

- **@openmaic/storage** separates storage contracts from implementations through the `KVStore` interface and `DeviceSafeKVStore` branding.
- **Scope enforcement** prevents `device`-local data from leaking to remote servers via compile-time types and runtime assertions in [`src/kv/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/kv/types.ts).
- **BrowserKVStore** provides `localStorage`-backed persistence with JSON validation and automatic namespacing.
- **HttpKVStore** enables account synchronization while rejecting device-scoped writes at the network boundary.
- **kvPersistStorage** offers type-safe Zustand integration for React applications, enforcing scope constraints through TypeScript overloads.
- **BrowserAssetStore** delivers content-addressed binary storage with reference counting, transaction safety, and URL lifecycle management via IndexedDB.
- **Pluggable backends** allow swapping between browser, HTTP, PostgreSQL, and S3 implementations without modifying business logic.

## Frequently Asked Questions

### What is the difference between `account` and `device` scope in @openmaic/storage?

The `account` scope designates data intended for cross-device synchronization, such as user preferences or cloud documents, while the `device` scope restricts data to the local machine for security-sensitive information like encryption keys or session tokens. As defined in [`src/kv/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/kv/types.ts), the `DeviceSafeKVStore` interface ensures that only backends with `servesDeviceScopeLocally: true` (such as `BrowserKVStore`) can handle `device`-scoped operations.

### How does @openmaic/storage prevent sensitive data from syncing to the cloud?

The package implements a two-layer defense: TypeScript branding via `DeviceSafeKVStore` prevents compilation of code attempting to pass non-local stores for `device` scope, while runtime assertions in `assertKVScope` throw `KVScopeViolationError` if scope literals are misused. Additionally, `HttpKVStore` explicitly rejects `device`-scoped writes, ensuring that local data never traverses network boundaries.

### Can I use @openmaic/storage with state management libraries other than Zustand?

While the package provides a first-party adapter via `kvPersistStorage` in [`src/zustand/persist.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/src/zustand/persist.ts), the underlying `KVStore` interface is framework-agnostic. Any library supporting asynchronous storage adapters (such as Redux Persist or Pinia) can consume `BrowserKVStore`, `HttpKVStore`, or `PgDocumentStore` directly, provided the adapter maps the store's `get`, `set`, and `remove` methods to the library's expected contract.

### How does BrowserAssetStore handle duplicate binary data?

`BrowserAssetStore` calculates content hashes for all incoming blobs and stores bytes in a dedicated `blobs` object store indexed by hash. When storing duplicate content, the registry creates a new `AssetId` pointing to the existing hash without writing redundant data. Reference counting within IndexedDB transactions ensures that shared blob entries are only deleted when all referencing assets are removed, optimizing storage for duplicate images or documents.