How Server‑Backed Deployment in OpenMAIC Handles Persistence

Server‑backed deployment in OpenMAIC uses compile‑time feature flags to swap client‑side IndexedDB storage for HTTP‑based stores that persist data to a central PostgreSQL database, authenticated via configurable tokens.

OpenMAIC is an open‑source multimodal AI composition framework developed by THU‑MAIC. When deploying OpenMAIC in production or collaborative environments, developers often need data to survive browser refreshes and be shared across users. This article explains exactly how the server‑backed persistence layer works, including the configuration flags, authentication mechanism, and database integration.

Enabling Server Persistence via Build‑Time Flags

OpenMAIC does not detect server persistence at runtime. Instead, it relies on Next.js public environment variables that are baked into the client bundle during next build.

Set these two variables to activate server‑backed mode:

NEXT_PUBLIC_PERSISTENCE=1
NEXT_PUBLIC_PERSISTENCE_TOKEN=openmaic-local-dev  # development token

The lib/persistence/bootstrap.ts module checks process.env.NEXT_PUBLIC_PERSISTENCE to decide which store implementations to instantiate【#lib/persistence/bootstrap.ts】. When the flag equals "1", the bootstrapper creates:

• HttpRuntimeStore – persists learner runtime state
• HttpDocumentStore – persists document compositions
• HttpAssetStore – persists uploaded media assets

If the flag is absent or falsy, the system falls back to IndexedDBStore variants that live entirely in the browser.

The HTTP Store Architecture

Server‑backed persistence is implemented as a thin HTTP client layer that mirrors the local IndexedDB API. This design lets the rest of the application remain agnostic to where data lives.

Client‑Side HTTP Stores

Each store class makes authenticated fetch() calls to /api/persistence/* endpoints:

Store Class	Endpoint Pattern	Purpose
HttpRuntimeStore	/api/persistence/runtime/*	Save/restore learner execution state
HttpDocumentStore	/api/persistence/documents/*	CRUD operations on multimodal documents
HttpAssetStore	/api/persistence/assets/*	Binary upload/download for media files

All requests include the x‑learner‑key header set to NEXT_PUBLIC_PERSISTENCE_TOKEN【#packages/@openmaic/storage/docs/runtime-http-contract.md】.

Server‑Side Route Handler

The API route at app/api/persistence/[...path]/route.ts acts as a unified gateway. It:

1. Validates the incoming x‑learner‑key token
2. Imports the server provider from lib/persistence/server-provider.ts
3. Delegates the operation to the appropriate PostgreSQL‑backed store

// app/api/persistence/[...path]/route.ts (conceptual)
export async function POST(request: Request) {
  const auth = await verifyPersistenceToken(request);
  if (!auth) return new Response('Unauthorized', { status: 401 });
  
  const { getServerPersistenceProvider } = await import('@/lib/persistence/server-provider');
  const provider = await getServerPersistenceProvider();
  
  // Route based on URL path segments...
}

PostgreSQL Storage Provider

The lib/persistence/server-provider.ts file lazily initializes concrete storage implementations from the @openmaic/storage package【#lib/persistence/server-provider.ts】:

// lib/persistence/server-provider.ts
export async function getServerPersistenceProvider() {
  const { PgRuntimeStore, PgDocumentStore, PgAssetStore } = await import('@openmaic/storage/pg');
  
  return {
    runtimeStore: new PgRuntimeStore(process.env.DATABASE_URL),
    documentStore: new PgDocumentStore(process.env.DATABASE_URL),
    assetStore: new PgAssetStore(process.env.DATABASE_URL),
  };
}

Each Pg*Store class implements the same interface as its IndexedDB counterpart, ensuring drop‑in compatibility. The storage layer handles connection pooling, migrations, and binary streaming for assets.

Authentication and Security Model

OpenMAIC uses a dual‑mode authentication strategy:

Development Mode

The default NEXT_PUBLIC_PERSISTENCE_TOKEN=openmaic-local-dev is a hardcoded shared secret. The server validates it in lib/persistence/server-auth.ts using a simple equality check. This is sufficient for local Docker Compose setups where exposure is limited.

Production Mode

For production deployments, you replace the dev token with a session‑based or JWT verifier. The auth module exports a pluggable interface:

// lib/persistence/server-auth.ts
export async function verifyPersistenceToken(req: Request): Promise<boolean> {
  if (process.env.NODE_ENV === 'development') {
    return req.headers.get('x-learner-key') === process.env.PERSISTENCE_DEV_TOKEN;
  }
  // Production: integrate with your auth provider (Clerk, Auth0, custom JWT, etc.)
  return verifyProductionSession(req);
}

Token validation failures return HTTP 401, causing the client to gracefully degrade to local IndexedDB storage with a console warning.

Docker Compose Deployment

The repository includes a server-persistence Docker Compose profile that wires everything together【#README.md】:

NEXT_PUBLIC_PERSISTENCE=1 \
NEXT_PUBLIC_PERSISTENCE_TOKEN=openmaic-local-dev \
docker compose --profile server-persistence up --build

This command:

1. Builds the Next.js app with server persistence enabled
2. Starts a PostgreSQL 15 container with persistent volume
3. Injects DATABASE_URL pointing to the running database
4. Exposes the persistence API on the same port as the main application

Environment Variable Reference

Variable	Required	Description
NEXT_PUBLIC_PERSISTENCE	Yes	Set to 1 to enable HTTP stores
NEXT_PUBLIC_PERSISTENCE_TOKEN	Yes	Client‑side token sent in x‑learner‑key header
DATABASE_URL	Server‑only	PostgreSQL connection string (e.g., postgres://user:pass@localhost:5432/openmaic)
PERSISTENCE_DEV_TOKEN	Optional	Server‑side validation override for local development

HTTP Contract Documentation

The storage package defines formal contracts for each store type, ensuring client and server remain synchronized during upgrades:

• Runtime store – packages/@openmaic/storage/docs/runtime-http-contract.md【runtime-http-contract.md】
• Document store – packages/@openmaic/storage/docs/document-http-contract.md【document-http-contract.md】
• Asset store – packages/@openmaic/storage/docs/asset-http-contract.md【asset-http-contract.md】

These specifications include exact endpoint paths, HTTP methods, request/response schemas, and error codes.

Client Integration Example

Here is a complete flow showing how a document is saved with server persistence:

// Client code (runs in browser)
async function saveDocument(stageId: string, content: DocumentContent) {
  const response = await fetch(`/api/persistence/documents/${stageId}`, {
    method: 'PUT',
    headers: {
      'Content-Type': 'application/json',
      'x-learner-key': process.env.NEXT_PUBLIC_PERSISTENCE_TOKEN!,
    },
    body: JSON.stringify({ content, updatedAt: Date.now() }),
  });
  
  if (response.status === 401) {
    console.warn('Server persistence unavailable, falling back to local storage');
    // Client automatically falls back to IndexedDB via bootstrap logic
  }
  
  return response.ok;
}

The server handler receives this request, validates the token, extracts the stageId from the URL, and calls provider.documentStore.save(stageId, body) to persist to PostgreSQL.

Fallback Behavior When Server Is Unavailable

The lib/persistence/bootstrap.ts module includes detection logic for network failures and authentication errors. If any HTTP store operation receives a 401, 403, or 5xx response, the system:

1. Logs a warning to the console
2. Silently switches to IndexedDBStore for the current session
3. Displays a non‑blocking UI indicator (if the application configures one)

This ensures users never lose work due to temporary server outages or misconfigured tokens.

Summary

• Compile‑time activation – NEXT_PUBLIC_PERSISTENCE=1 bakes server‑backed mode into the client bundle at build time
• HTTP store abstraction – HttpRuntimeStore, HttpDocumentStore, and HttpAssetStore provide identical APIs to their IndexedDB counterparts
• Token‑based authentication – NEXT_PUBLIC_PERSISTENCE_TOKEN secures requests; pluggable for production auth systems
• PostgreSQL backend – lib/persistence/server-provider.ts instantiates @openmaic/storage/pg implementations
• Graceful degradation – Network or auth failures automatically fall back to local browser storage

Frequently Asked Questions

Can I switch from server‑backed to local persistence without rebuilding?

No. NEXT_PUBLIC_PERSISTENCE is a build‑time constant processed by Next.js. Changing persistence modes requires recompiling the application. For dynamic switching, you would need to maintain both store implementations and add runtime routing logic.

What database migrations does OpenMAIC require?

The @openmaic/storage package includes a migrate CLI command that runs automatically on server startup in development. For production, run npx @openmaic/storage migrate against your DATABASE_URL before deploying. Migrations create tables for runtime snapshots, document versions, and asset blobs with proper indexing.

How do I rotate the persistence token in production?

Replace PERSISTENCE_DEV_TOKEN with a production secret manager. Update lib/persistence/server-auth.ts to validate tokens against your identity provider (JWT, session cookies, or API keys). Clients receive new tokens through your standard authentication flow—no changes to NEXT_PUBLIC_PERSISTENCE_TOKEN needed if you implement session‑based verification.

Is server persistence required for multi‑user collaboration?

Not strictly, but it is strongly recommended. Without server persistence, each user maintains isolated IndexedDB state. Real‑time collaboration features (WebRTC sync, presence cursors) can work peer‑to‑peer, but data durability and cross‑device access require the PostgreSQL backend that server persistence provides.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}