# How Persistence Is Wired for Browser and Server in OpenMAIC's lib/persistence > Discover how OpenMAIC's lib/persistence wires browser and server for transparent data storage. Learn about HTTP-backed stores and direct PostgreSQL connections. - Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC) - Tags: internals - Published: 2026-09-02 --- **OpenMAIC uses environment-specific wiring in `lib/persistence` to deliver transparent data storage: browser-context code configures HTTP-backed stores that forward to a server-side PostgreSQL provider, while server-context code connects directly to the database via a cached singleton factory.** The **OpenMAIC** repository implements a **dual-environment persistence architecture** that lets the same application code run with in-memory, localStorage, or full PostgreSQL backing depending on configuration. The `lib/persistence` directory contains the bootstrap logic and provider factories that make this abstraction seamless. --- ## Browser-Side Persistence Wiring Client-side persistence is orchestrated from **[`lib/persistence/bootstrap.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/bootstrap.ts)**. This module detects the runtime environment and conditionally replaces default in-memory stores with HTTP-backed equivalents. ### Feature Flag Detection The bootstrap checks **`NEXT_PUBLIC_PERSISTENCE`** to determine if persistence should activate: ```ts // lib/persistence/bootstrap.ts function isBrowserPersistenceEnabled(): boolean { return process.env.NEXT_PUBLIC_PERSISTENCE === '1'; } ``` Without this flag, the module exports no-op configurations and the application uses purely in-memory storage. ### HTTP Store Construction When enabled, the bootstrap creates three key components: - **`BrowserKVStore`** — A thin wrapper over `localStorage` that stores the **learner key** (a per-device identifier) - **`HttpRuntimeStore`** — Forwards runtime state CRUD to `/api/persistence` - **`HttpDocumentStore`** — Forwards document operations to the same endpoint ```ts // lib/persistence/bootstrap.ts if (isBrowserPersistenceEnabled()) { const learnerKey = getPersistenceLearnerKey; // Lazy promise resolving to localStorage value const headers = getPersistenceRequestHeaders; // Includes learner key + optional auth token configureRuntimeStorage({ store: () => new HttpRuntimeStore({ baseUrl: '/api/persistence', headers }), learnerKey, }); configureDocumentStorage({ store: ({ validateScene, validateStage }) => new HttpDocumentStore({ baseUrl: '/api/persistence', headers, validateScene, validateStage, }), }); } ``` The **`configureRuntimeStorage`** and **`configureDocumentStorage`** functions (from `lib/runtime/config` and `lib/document-store/config`) replace the default store implementations at module load time. This allows all other application code to import the generic `RuntimeStore` and `DocumentStore` interfaces without conditional logic. --- ## Server-Side Persistence Wiring Server-side persistence is managed by **[`lib/persistence/server-provider.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/server-provider.ts)**, which exposes **`getServerPersistenceProvider(connectionString)`** — a singleton factory that returns a PostgreSQL-backed provider. ### Singleton Provider Pattern The factory uses a **Symbol-keyed global cache** to avoid creating multiple connection pools across serverless invocations: ```ts // lib/persistence/server-provider.ts const PROVIDER_SYMBOL = Symbol.for('openmaic.persistence.provider'); interface ProviderState { providerPromise?: Promise; connectionString?: string; } const providerState: ProviderState = (globalThis as any)[PROVIDER_SYMBOL] ?? {}; export async function getServerPersistenceProvider( connectionString: string, poolFactory = (cs: string) => new Pool({ connectionString: cs }) ): Promise { // Return cached instance if connection string matches if (providerState.providerPromise && providerState.connectionString === connectionString) { return providerState.providerPromise; } // Create fresh provider with PostgreSQL stores const provider = await createServerPersistenceProvider( connectionString, poolFactory ); // Cache for subsequent calls providerState.providerPromise = Promise.resolve(provider); providerState.connectionString = connectionString; (globalThis as any)[PROVIDER_SYMBOL] = providerState; return provider; } ``` ### PostgreSQL Store instantiation Inside **`createServerPersistenceProvider`**, the factory instantiates three concrete stores from **`@openmaic/storage`**: | Store | Purpose | Key Configuration | |-------|---------|-------------------| | `PgRuntimeStore` | Runtime state persistence | Transaction wrapper, payload validators | | `PgDocumentStore` | Document CRUD operations | Scene/stage validation hooks | | `PgAssetStore` | Binary asset storage | Transaction wrapper, byte store backend | ```ts // Inside createServerPersistenceProvider() const runtimeStore = new PgRuntimeStore(queryable, { withTransaction, payloadValidators: APP_RUNTIME_PAYLOAD_VALIDATORS, }); const documentStore = new PgDocumentStore(queryable, { withTransaction, validateScene: validateAppScene, validateStage: validateAppStage, }); const assetStore = new PgAssetStore(queryable, { withTransaction, byteStore: createByteStore() }); // Ensure database schemas exist await ensureSchema(queryable); await ensureDocumentSchema(queryable); ``` The **validation hooks** (`validateAppScene`, `validateAppStage`) enforce application-specific invariants before write operations, while **`ensureSchema`** and **`ensureDocumentSchema`** create tables and indexes on first connection. --- ## Connecting Browser to Server: The HTTP Bridge The HTTP persistence API (**`pages/api/persistence/*`**) acts as the bridge between client and server stores. Route handlers obtain the provider via `getServerPersistenceProvider` and delegate to the appropriate PostgreSQL store: ```ts // Conceptual route handler structure import { getServerPersistenceProvider } from '@/lib/persistence/server-provider'; export default async function handler(req, res) { const provider = await getServerPersistenceProvider( process.env.DATABASE_URL! ); // Route to appropriate store based on path const { data, error } = await provider.runtimeStore[req.method.toLowerCase()]( req.body ); res.status(error ? 500 : 200).json({ data, error }); } ``` The **`learnerKey`** header sent by browser stores enables the server to isolate data per device/user without requiring authentication overhead for every operation. --- ## Key Supporting Files Several additional modules in `lib/persistence` handle cross-cutting concerns: - **[`lib/persistence/document-access.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/document-access.ts)** — Parses HTTP methods into document actions (`create`, `read`, `update`, `delete`) and implements permission logic using stage metadata - **[`lib/persistence/plain-json.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/plain-json.ts)** — Canonicalizes JavaScript objects to deterministic JSON strings before persistence (ensures hash stability) - **[`lib/persistence/asset-byte-store.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/asset-byte-store.ts)** — Lazily initializes S3-compatible storage for large binary assets - **[`lib/persistence/stage-meta.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/stage-meta.ts)** — Defines PostgreSQL schema for stage ownership, deletion tombstones, and access control - **[`lib/persistence/owner-materials.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/owner-materials.ts)** — Schema definition for user-specific learning materials --- ## Practical Examples ### Enabling Persistence in Development ```bash # .env.local NEXT_PUBLIC_PERSISTENCE=1 DATABASE_URL=postgresql://localhost:5432/openmaic ``` ### Using Stores in React Components ```tsx import { useRuntimeStore } from '@/lib/runtime/store'; function ProgressTracker() { const runtime = useRuntimeStore(); // Automatically persists to PostgreSQL via HTTP when flag is enabled const markComplete = () => { runtime.set('lessonCompleted', true); }; return ; } ``` ### Direct Database Access in Server Code ```ts import { getServerPersistenceProvider } from '@/lib/persistence/server-provider'; async function migrateStageOwnership(oldOwnerId: string, newOwnerId: string) { const provider = await getServerPersistenceProvider( process.env.DATABASE_URL! ); const stages = await provider.documentStore.list({ ownerId: oldOwnerId }); for (const stage of stages) { await provider.stageMeta.transferOwnership(stage.id, newOwnerId); } } ``` ### Document Access Control ```ts import { parseDocumentAction, decideDocumentAccess } from '@/lib/persistence/document-access'; import { getServerPersistenceProvider } from '@/lib/persistence/server-provider'; async function authorizeRequest(req: Request): Promise { const action = parseDocumentAction(req.method, req.url); const ownerId = req.headers.get('x-owner-id'); const provider = await getServerPersistenceProvider( process.env.DATABASE_URL! ); const access = await decideDocumentAccess( action, ownerId, (id) => provider.stageMeta.read(id), (id) => provider.documentStore.exists(id) ); return access === 'allow'; } ``` --- ## Summary - **[`lib/persistence/bootstrap.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/bootstrap.ts)** conditionally enables HTTP-backed stores in the browser based on `NEXT_PUBLIC_PERSISTENCE` - **[`lib/persistence/server-provider.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/persistence/server-provider.ts)** creates a singleton PostgreSQL provider cached via `Symbol.for('openmaic.persistence.provider')` - HTTP stores (`HttpRuntimeStore`, `HttpDocumentStore`) forward client operations to server routes, which delegate to `PgRuntimeStore`, `PgDocumentStore`, and `PgAssetStore` - The **learner key** from `BrowserKVStore` provides request isolation without full authentication - Validation hooks and schema creation run on provider initialization to ensure database readiness - All application code uses generic store interfaces; the wiring layer handles environment-specific implementation selection --- ## Frequently Asked Questions ### How does OpenMAIC avoid database connection exhaustion in serverless environments? The **`getServerPersistenceProvider`** function caches the provider promise on `globalThis` using a Symbol key. Subsequent calls with the same connection string return the cached instance, preventing new `Pool` creation on every serverless invocation. This pattern is essential for platforms like Vercel where functions may cold-start frequently. ### What happens when `NEXT_PUBLIC_PERSISTENCE` is not set? The bootstrap module exports the default in-memory store configurations. All `RuntimeStore` and `DocumentStore` operations remain local to the browser tab, and no HTTP calls are made to `/api/persistence`. Data is lost on page refresh. ### Why does the browser use HTTP stores instead of connecting directly to PostgreSQL? Direct PostgreSQL connections from browsers require exposing credentials and allowing public network access to the database. The HTTP bridge in `pages/api/persistence` provides authentication, authorization via **`decideDocumentAccess`**, and request validation while keeping database credentials server-side only. ### How is the learner key generated and preserved? The **`getPersistenceLearnerKey`** function in [`bootstrap.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/bootstrap.ts) checks `BrowserKVStore` (wrapping `localStorage`) for an existing UUID. If absent, it generates a new v4 UUID and stores it. This key is included in all HTTP request headers, allowing the server to associate runtime state and documents with a specific device without requiring user login.