How the OpenMAIC Asset Pool Architecture Works with AssetIds and Content Hashes

OpenMAIC uses a browser-based asset pool backed by IndexedDB where opaque AssetId identifiers are allocated via toAssetId while content hashes enable duplicate detection and deduplication.

The OpenMAIC framework (THU-MAIC/OpenMAIC) manages all user-generated media—audio, video, images, and recordings—through a unified asset pool abstraction. This architecture separates the concerns of unique identification (via allocated IDs) from content verification (via cryptographic hashes), enabling efficient storage, cross-tab coordination, and intelligent cache management.

Asset Pool Foundation: Lazy Initialization and IndexedDB Storage

The asset pool is instantiated lazily through getAssetPool() in [lib/media/asset-pool.ts](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/media/asset-pool.ts). When first called, this function creates a BrowserAssetStore backed by an IndexedDB database named maic-asset-pool.

// Lazy pool creation pattern
import { getAssetPool } from '@/lib/media/asset-pool';

// First call initializes the IndexedDB store
const pool = getAssetPool();
// Subsequent calls return the same singleton instance

The pool implements the AssetPoolStore interface defined in [lib/media/asset-pool-config.ts](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/media/asset-pool-config.ts), providing three core operations:

• put(blob) — Stores a blob and returns an AssetId
• resolve(assetId) — Returns an Object URL for rendering
• remove(assetId) — Deletes the asset from storage

AssetId Generation: Opaque Identifiers via toAssetId

Asset IDs in OpenMAIC are allocated, not derived. The toAssetId helper from the @openmaic/storage package generates opaque identifiers that serve as primary keys in IndexedDB.

Based on the package structure, the ID generation logic resides in:

• Path: packages/@openmaic/storage/src/asset/id.ts
• Function: toAssetId(input: unknown): AssetId

// Conceptual usage based on architecture
import { toAssetId } from '@openmaic/storage';

// Generates a unique, opaque identifier
const assetId = toAssetId(someInput);
// Result: "asset-12345" or similar UUID-derived string

Key characteristics of AssetIds:

• Opaque — No semantic meaning embedded in the ID itself
• Unique — Guaranteed unique across browser sessions via allocation strategy
• Persistent — Survives page reloads because stored as IndexedDB primary keys
• Not content-derived — Same blob added twice receives different IDs unless deduplication intervenes

Content Hashes: SHA-256 for Duplicate Detection

While AssetIds handle uniqueness, content hashes (typically SHA-256) enable deterministic duplicate detection. The hash is computed once during blob ingestion and stored as asset metadata.

Hash Computation and Storage

When BrowserAssetStore.put() processes a new blob:

1. Compute content hash of the blob bytes
2. Generate opaque AssetId via toAssetId
3. Store tuple: (AssetId, contentHash, blobData) in IndexedDB

The hash lives in metadata, not as the primary key, allowing flexible deduplication strategies.

Deduplication in Practice

Higher-level collectors implement hash-based reuse. The test suite in [tests/video-export/collect.test.ts](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/video-export/collect.test.ts) demonstrates this pattern:

// Simplified deduplication workflow
import { hashBlob } from '@/lib/utils/hash';

async function storeWithDeduplication(newBlob: Blob) {
  const hash = await hashBlob(newBlob);
  
  // Query pool for existing hash match
  const existing = await getAssetPool().findByHash(hash);
  
  if (existing) {
    // Reuse existing asset—saves storage
    return existing.id;
  }
  
  // No match—store as new asset
  return await getAssetPool().put(newBlob);
}

Complete Asset Lifecycle

Phase	Operation	Implementation
Initialization	Create/retrieve pool	getAssetPool() → BrowserAssetStore constructor
Ingestion	Add blob with ID + hash	pool.put() → toAssetId() + hash computation
Resolution	Get renderable URL	pool.resolve() → blob: Object URL
Deduplication	Reuse by hash match	Collector-level hash comparison
Eviction	Remove single asset	pool.remove(assetId)
Clearing	Destroy entire pool	clearAssetPool() with deferred deletion handling

Cross-Tab Coordination and Asset Replacement

The pool architecture includes mechanisms for concurrent editing scenarios:

• observeAssetReplacements — Invalidates URL cache when assets are replaced
• bindAssetReplacementChannel — Presence channel for probing replacement safety across tabs

These features ensure that when one tab updates an asset, other tabs automatically reload the corrected version rather than serving stale cached URLs.

Code Examples

Basic Asset Storage and Retrieval

import { getAssetPool } from '@/lib/media/asset-pool';

async function handleRecording(blob: Blob) {
  // Store and receive opaque ID
  const assetId = await getAssetPool().put(blob);
  console.log('Stored as:', assetId); // e.g., "asset-a1b2c3d4"
  
  // Resolve for rendering
  const objectUrl = await getAssetPool().resolve(assetId);
  const videoElement = document.createElement('video');
  videoElement.src = objectUrl;
  
  return { assetId, objectUrl };
}

Cache Management Workflow

import { clearAssetPool } from '@/lib/media/asset-pool';

async function clearAllCache() {
  try {
    await clearAssetPool();
    console.log('Asset pool cleared');
  } catch (error) {
    if (error.name === 'AssetPoolDeletionDeferredError') {
      // Pool in use elsewhere—retry scheduled
      console.log('Deletion deferred');
    }
    throw error;
  }
}

React Hook for Asset URLs

// From lib/media/use-asset-url.ts
import { useAssetUrl } from '@/lib/media/use-asset-url';

function VideoPlayer({ assetId }: { assetId: string }) {
  const { url, loading, error } = useAssetUrl(assetId);
  
  if (loading) return <Spinner />;
  if (error) return <ErrorMessage />;
  
  return <video src={url} controls />;
}

Summary

• AssetIds are allocated via toAssetId in @openmaic/storage, providing opaque, unique identifiers for IndexedDB primary keys
• Content hashes are computed separately (SHA-256) and stored as metadata to enable duplicate detection without conflating identity with content
• The asset pool (getAssetPool() / BrowserAssetStore) abstracts IndexedDB operations behind a clean put/resolve/remove interface
• Deduplication happens at the collector level, checking hash matches before storing new blobs
• Cross-tab coordination ensures asset replacements propagate correctly via presence channels and replacement observers

Frequently Asked Questions

What is the difference between an AssetId and a content hash in OpenMAIC?

An AssetId is an opaque, allocated string generated by toAssetId that uniquely identifies a stored asset in IndexedDB. A content hash is a deterministic SHA-256 digest computed from the blob's bytes. The ID provides a stable reference for the stored object, while the hash enables detecting whether two blobs contain identical content.

Why doesn't OpenMAIC use the content hash as the primary key?

Using allocated IDs rather than content hashes as primary keys allows the same logical content to exist with multiple identities when needed (different metadata, processing pipelines, or ownership contexts). It also decouples storage addressing from content validation, letting the system evolve hash algorithms or support hash-agnostic operations without migrating primary keys.

How does the asset pool handle storage when the same file is uploaded twice?

Without deduplication logic, the second upload receives a new AssetId and stores a duplicate copy. With deduplication enabled (as in video export collectors), the system computes the hash, finds the existing match via findByHash, and returns the original AssetId instead of storing redundant data.

What happens to AssetIds when the browser cache is cleared?

clearAssetPool() closes the IndexedDB connection and deletes the entire maic-asset-pool database. All AssetId-to-blob mappings are irrevocably lost. Applications using OpenMAIC must handle AssetPoolDeletionDeferredError when the pool is locked by another tab, scheduling retry attempts until the operation succeeds.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}