How OpenMAIC Handles Document Validation and Migration Using Zod Validators

OpenMAIC validates documents at write-time through validateAppScene and validateAppStage functions, then re-validates them during a three-phase lazy migration pipeline to ensure legacy data safely converts to the current DSL schema without corruption.

OpenMAIC stores user-generated content—including scenes, stages, and outlines—in a document store backed by IndexedDB. To guarantee data integrity across application versions, the repository implements a validation strategy using Zod validators that check structural conformity both when documents are persisted and when they are lazily migrated from legacy formats.

Write-Time Validation with Zod

When an application writes new content, OpenMAIC intercepts the operation to validate payload structure before it reaches storage.

The Validation Layer in validators.ts

The core validation logic resides in lib/document-store/validators.ts. This module imports Zod-generated type guards—including isActionType, validateAction, validateScene, and validateStage—from the @openmaic/dsl package. It composes these primitives into higher-order validators specific to the application's extended scene unions (interactive scenes, PBL scenes, etc.).

The validation functions return a structured payload:

{ 
  valid: boolean, 
  errors?: ValidationIssue[] 
}

This pattern allows both the document store and migration engine to consume validation results programmatically, aborting operations when structural checks fail.

Validating Scenes and Stages

validateAppScene and validateAppStage serve as the primary entry points for write-time validation. When a user creates or updates a scene through the UI, the application invokes these functions to verify that the payload contains non-empty strings, numeric ordering, and matching type/content.type pairs. If validation fails, the write aborts before IndexedDB storage occurs, preventing corrupt data from entering the document store.

Migration-Time Verification

OpenMAIC employs lazy migration to upgrade legacy documents on demand rather than batch-processing stored data. The migration engine runs the same Zod validators on both source analysis and destination output to guarantee conversion accuracy.

The Three-Phase Migration Pipeline

The migrateLocked function in lib/document-store/migration.ts orchestrates document migration through three distinct phases, using validators at critical checkpoints:

1. Probe Phase: Reads the current destination document and legacy snapshot. If a destination exists, assertValidDestination invokes validateAppStage and validateAppScene to confirm the stored document already conforms to the current schema.
2. Convert Phase: Transforms legacy media references into allocated asset IDs without holding a lock. No validation occurs here because the conversion may produce intermediate structures.
3. Commit Phase: Re-acquires the lock, reconciles concurrent edits, and persists the converted document. After conversion, assertValidDestination runs again on the new document. If the migration produced a canonical document from the legacy snapshot, assertMigrationVerified re-runs validators to confirm the DSL migration (migrate) produced the same structural shape as the canonical snapshot.

Migration Markers and Data Safety

Once validation succeeds, the migration machinery records a marker in IndexedDB using the key pattern document-migration:<stageId>. This prevents redundant migration work on subsequent accesses. If any validator reports an error during the pipeline, the migration aborts entirely, leaving the legacy document untouched to preserve data safety.

Validation Strategy and Error Handling

The validators implement a deliberate balance between backward compatibility and structural rigor.

Lenient on History, Strict on Structure

OpenMAIC's validation layer is lenient on historical data—allowing unknown action types to pass through—while remaining strict on required fields such as non-empty strings, numeric order fields, and type discriminator matching. This approach enables older documents to survive schema evolution without manual intervention, while still blocking payloads that would crash the playback engine due to missing critical fields.

Runtime Integration

The accessDocument and mutateDocument functions exported from lib/document-store/store.ts provide the high-level API for document interaction. These functions automatically invoke the migration pipeline when detecting legacy data, ensuring that callers always receive Zod-validated documents regardless of when the data was originally created.

Practical Implementation Examples

Validating a Scene Before Persistence

import { validateAppScene } from '@/lib/document-store/validators';

const scene = {/* …scene data from UI… */};
const result = validateAppScene(scene);

if (!result.valid) {
  console.error('Scene validation failed', result.errors);
  // Reject the write or correct the payload
}

Loading a Document with Automatic Migration

import { accessDocument } from '@/lib/document-store/migration';

async function openStage(stageId: string) {
  const { document, readOnlyLegacy } = await accessDocument(stageId);
  
  if (document) {
    console.log('Document ready', document);
    // `document` has passed Zod validation already
  } else if (readOnlyLegacy) {
    console.warn('Stage only has legacy data – migration deferred');
  }
}

Mutating a Document Safely

import { mutateDocument } from '@/lib/document-store/migration';
import { validateAppScene } from '@/lib/document-store/validators';

async function addScene(stageId: string, newScene) {
  await mutateDocument(stageId, async (doc, store) => {
    const validated = validateAppScene(newScene);
    if (!validated.valid) throw new Error('Invalid scene');

    const updated = doc 
      ? { ...doc, scenes: [...doc.scenes, newScene] } 
      : { stage: {}, scenes: [newScene] };
    
    await store.saveDocument(updated);
  });
}

Summary

• OpenMAIC validates documents at two critical points: write-time (via validateAppScene and validateAppStage) and migration-time (via assertValidDestination and assertMigrationVerified).
• The validation implementation in lib/document-store/validators.ts imports Zod primitives from @openmaic/dsl and extends them for application-specific scene unions.
• The migrateLocked routine in lib/document-store/migration.ts executes a three-phase pipeline (Probe, Convert, Commit) that validates both source and destination documents before persisting changes.
• Failed validation immediately aborts migration, preserving legacy data and recording migration markers (document-migration:<stageId>) only after successful verification.
• Validators tolerate unknown historical fields while enforcing strict requirements on critical structural fields, preventing runtime crashes while maintaining backward compatibility.

Frequently Asked Questions

What is the validation payload format returned by OpenMAIC validators?

OpenMAIC validators return an object with the shape { valid: boolean, errors?: ValidationIssue[] }. The valid boolean indicates structural conformity, while the optional errors array contains detailed descriptions of validation failures that the migration engine or UI can consume for error reporting.

How does OpenMAIC prevent data loss during document migration?

The migration pipeline in lib/document-store/migration.ts performs validation checks in the Probe and Commit phases using assertValidDestination and assertMigrationVerified. If Zod validation fails at any checkpoint, the operation aborts before overwriting legacy data. The system only records migration markers in IndexedDB after successful validation, ensuring that corrupt conversions never replace valid legacy snapshots.

Where does the Zod schema originate in OpenMAIC?

The base Zod schemas and type guards—such as isActionType, validateAction, validateScene, and validateStage—originate from the @openmaic/dsl package. The lib/document-store/validators.ts file imports these primitives and composes them into higher-level functions like validateAppScene that handle the application's extended scene types (interactive, PBL, etc.).

How does lazy migration work in OpenMAIC?

OpenMAIC implements lazy migration through the accessDocument function, which checks for legacy data when loading a stage. If legacy data exists, the system invokes migrateLocked to convert the document on demand rather than batch-processing stored documents. The conversion occurs in three phases (Probe, Convert, Commit), with Zod validators verifying the output before the function returns the document to the caller, ensuring that the application always works with current-schema data.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}