How Persistence Is Wired for Browser and Server in OpenMAIC's lib/persistence

OpenMAIC uses environment-specific wiring in lib/persistence to deliver transparent data storage: browser-context code configures HTTP-backed stores that forward to a server-side PostgreSQL provider, while server-context code connects directly to the database via a cached singleton factory.

The OpenMAIC repository implements a dual-environment persistence architecture that lets the same application code run with in-memory, localStorage, or full PostgreSQL backing depending on configuration. The lib/persistence directory contains the bootstrap logic and provider factories that make this abstraction seamless.

---

Browser-Side Persistence Wiring

Client-side persistence is orchestrated from lib/persistence/bootstrap.ts. This module detects the runtime environment and conditionally replaces default in-memory stores with HTTP-backed equivalents.

Feature Flag Detection

The bootstrap checks NEXT_PUBLIC_PERSISTENCE to determine if persistence should activate:

// lib/persistence/bootstrap.ts
function isBrowserPersistenceEnabled(): boolean {
  return process.env.NEXT_PUBLIC_PERSISTENCE === '1';
}

Without this flag, the module exports no-op configurations and the application uses purely in-memory storage.

HTTP Store Construction

When enabled, the bootstrap creates three key components:

• BrowserKVStore — A thin wrapper over localStorage that stores the learner key (a per-device identifier)
• HttpRuntimeStore — Forwards runtime state CRUD to /api/persistence
• HttpDocumentStore — Forwards document operations to the same endpoint

// lib/persistence/bootstrap.ts
if (isBrowserPersistenceEnabled()) {
  const learnerKey = getPersistenceLearnerKey;   // Lazy promise resolving to localStorage value
  const headers = getPersistenceRequestHeaders;  // Includes learner key + optional auth token

  configureRuntimeStorage({
    store: () => new HttpRuntimeStore({ 
      baseUrl: '/api/persistence', 
      headers 
    }),
    learnerKey,
  });

  configureDocumentStorage({
    store: ({ validateScene, validateStage }) => 
      new HttpDocumentStore({
        baseUrl: '/api/persistence',
        headers,
        validateScene,
        validateStage,
      }),
  });
}

The configureRuntimeStorage and configureDocumentStorage functions (from lib/runtime/config and lib/document-store/config) replace the default store implementations at module load time. This allows all other application code to import the generic RuntimeStore and DocumentStore interfaces without conditional logic.

---

Server-Side Persistence Wiring

Server-side persistence is managed by lib/persistence/server-provider.ts, which exposes getServerPersistenceProvider(connectionString) — a singleton factory that returns a PostgreSQL-backed provider.

Singleton Provider Pattern

The factory uses a Symbol-keyed global cache to avoid creating multiple connection pools across serverless invocations:

// lib/persistence/server-provider.ts
const PROVIDER_SYMBOL = Symbol.for('openmaic.persistence.provider');

interface ProviderState {
  providerPromise?: Promise<ServerPersistenceProvider>;
  connectionString?: string;
}

const providerState: ProviderState = 
  (globalThis as any)[PROVIDER_SYMBOL] ?? {};

export async function getServerPersistenceProvider(
  connectionString: string,
  poolFactory = (cs: string) => new Pool({ connectionString: cs })
): Promise<ServerPersistenceProvider> {
  
  // Return cached instance if connection string matches
  if (providerState.providerPromise 
      && providerState.connectionString === connectionString) {
    return providerState.providerPromise;
  }

  // Create fresh provider with PostgreSQL stores
  const provider = await createServerPersistenceProvider(
    connectionString, 
    poolFactory
  );

  // Cache for subsequent calls
  providerState.providerPromise = Promise.resolve(provider);
  providerState.connectionString = connectionString;
  (globalThis as any)[PROVIDER_SYMBOL] = providerState;

  return provider;
}

PostgreSQL Store instantiation

Inside createServerPersistenceProvider, the factory instantiates three concrete stores from @openmaic/storage:

Store	Purpose	Key Configuration
PgRuntimeStore	Runtime state persistence	Transaction wrapper, payload validators
PgDocumentStore	Document CRUD operations	Scene/stage validation hooks
PgAssetStore	Binary asset storage	Transaction wrapper, byte store backend

// Inside createServerPersistenceProvider()
const runtimeStore = new PgRuntimeStore(queryable, {
  withTransaction,
  payloadValidators: APP_RUNTIME_PAYLOAD_VALIDATORS,
});

const documentStore = new PgDocumentStore(queryable, {
  withTransaction,
  validateScene: validateAppScene,
  validateStage: validateAppStage,
});

const assetStore = new PgAssetStore(queryable, { 
  withTransaction, 
  byteStore: createByteStore() 
});

// Ensure database schemas exist
await ensureSchema(queryable);
await ensureDocumentSchema(queryable);

The validation hooks (validateAppScene, validateAppStage) enforce application-specific invariants before write operations, while ensureSchema and ensureDocumentSchema create tables and indexes on first connection.

---

Connecting Browser to Server: The HTTP Bridge

The HTTP persistence API (pages/api/persistence/*) acts as the bridge between client and server stores. Route handlers obtain the provider via getServerPersistenceProvider and delegate to the appropriate PostgreSQL store:

// Conceptual route handler structure
import { getServerPersistenceProvider } from '@/lib/persistence/server-provider';

export default async function handler(req, res) {
  const provider = await getServerPersistenceProvider(
    process.env.DATABASE_URL!
  );
  
  // Route to appropriate store based on path
  const { data, error } = await provider.runtimeStore[req.method.toLowerCase()](
    req.body
  );
  
  res.status(error ? 500 : 200).json({ data, error });
}

The learnerKey header sent by browser stores enables the server to isolate data per device/user without requiring authentication overhead for every operation.

---

Key Supporting Files

Several additional modules in lib/persistence handle cross-cutting concerns:

• lib/persistence/document-access.ts — Parses HTTP methods into document actions (create, read, update, delete) and implements permission logic using stage metadata
• lib/persistence/plain-json.ts — Canonicalizes JavaScript objects to deterministic JSON strings before persistence (ensures hash stability)
• lib/persistence/asset-byte-store.ts — Lazily initializes S3-compatible storage for large binary assets
• lib/persistence/stage-meta.ts — Defines PostgreSQL schema for stage ownership, deletion tombstones, and access control
• lib/persistence/owner-materials.ts — Schema definition for user-specific learning materials

---

Practical Examples

Enabling Persistence in Development

# .env.local

NEXT_PUBLIC_PERSISTENCE=1
DATABASE_URL=postgresql://localhost:5432/openmaic

Using Stores in React Components

import { useRuntimeStore } from '@/lib/runtime/store';

function ProgressTracker() {
  const runtime = useRuntimeStore();
  
  // Automatically persists to PostgreSQL via HTTP when flag is enabled
  const markComplete = () => {
    runtime.set('lessonCompleted', true);
  };
  
  return <button onClick={markComplete}>Complete Lesson</button>;
}

Direct Database Access in Server Code

import { getServerPersistenceProvider } from '@/lib/persistence/server-provider';

async function migrateStageOwnership(oldOwnerId: string, newOwnerId: string) {
  const provider = await getServerPersistenceProvider(
    process.env.DATABASE_URL!
  );
  
  const stages = await provider.documentStore.list({
    ownerId: oldOwnerId
  });
  
  for (const stage of stages) {
    await provider.stageMeta.transferOwnership(stage.id, newOwnerId);
  }
}

Document Access Control

import { parseDocumentAction, decideDocumentAccess } from '@/lib/persistence/document-access';
import { getServerPersistenceProvider } from '@/lib/persistence/server-provider';

async function authorizeRequest(req: Request): Promise<boolean> {
  const action = parseDocumentAction(req.method, req.url);
  const ownerId = req.headers.get('x-owner-id');
  
  const provider = await getServerPersistenceProvider(
    process.env.DATABASE_URL!
  );
  
  const access = await decideDocumentAccess(
    action,
    ownerId,
    (id) => provider.stageMeta.read(id),
    (id) => provider.documentStore.exists(id)
  );
  
  return access === 'allow';
}

---

Summary

• lib/persistence/bootstrap.ts conditionally enables HTTP-backed stores in the browser based on NEXT_PUBLIC_PERSISTENCE
• lib/persistence/server-provider.ts creates a singleton PostgreSQL provider cached via Symbol.for('openmaic.persistence.provider')
• HTTP stores (HttpRuntimeStore, HttpDocumentStore) forward client operations to server routes, which delegate to PgRuntimeStore, PgDocumentStore, and PgAssetStore
• The learner key from BrowserKVStore provides request isolation without full authentication
• Validation hooks and schema creation run on provider initialization to ensure database readiness
• All application code uses generic store interfaces; the wiring layer handles environment-specific implementation selection

---

Frequently Asked Questions

How does OpenMAIC avoid database connection exhaustion in serverless environments?

The getServerPersistenceProvider function caches the provider promise on globalThis using a Symbol key. Subsequent calls with the same connection string return the cached instance, preventing new Pool creation on every serverless invocation. This pattern is essential for platforms like Vercel where functions may cold-start frequently.

What happens when NEXT_PUBLIC_PERSISTENCE is not set?

The bootstrap module exports the default in-memory store configurations. All RuntimeStore and DocumentStore operations remain local to the browser tab, and no HTTP calls are made to /api/persistence. Data is lost on page refresh.

Why does the browser use HTTP stores instead of connecting directly to PostgreSQL?

Direct PostgreSQL connections from browsers require exposing credentials and allowing public network access to the database. The HTTP bridge in pages/api/persistence provides authentication, authorization via decideDocumentAccess, and request validation while keeping database credentials server-side only.

How is the learner key generated and preserved?

The getPersistenceLearnerKey function in bootstrap.ts checks BrowserKVStore (wrapping localStorage) for an existing UUID. If absent, it generates a new v4 UUID and stores it. This key is included in all HTTP request headers, allowing the server to associate runtime state and documents with a specific device without requiring user login.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}