# How OpenMAIC Handles Document Validation and Migration Using Zod Validators

> OpenMAIC ensures data integrity during document validation and migration using Zod. Discover how its write-time validation and lazy migration pipeline safely convert legacy data to the current DSL schema.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: how-to-guide
- Published: 2026-09-04

---

**OpenMAIC validates documents at write-time through `validateAppScene` and `validateAppStage` functions, then re-validates them during a three-phase lazy migration pipeline to ensure legacy data safely converts to the current DSL schema without corruption.**

OpenMAIC stores user-generated content—including scenes, stages, and outlines—in a document store backed by IndexedDB. To guarantee data integrity across application versions, the repository implements a validation strategy using **Zod** validators that check structural conformity both when documents are persisted and when they are lazily migrated from legacy formats.

## Write-Time Validation with Zod

When an application writes new content, OpenMAIC intercepts the operation to validate payload structure before it reaches storage.

### The Validation Layer in [`validators.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/validators.ts)

The core validation logic resides in **[`lib/document-store/validators.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/document-store/validators.ts)**. This module imports Zod-generated type guards—including `isActionType`, `validateAction`, `validateScene`, and `validateStage`—from the `@openmaic/dsl` package. It composes these primitives into higher-order validators specific to the application's extended scene unions (interactive scenes, PBL scenes, etc.).

The validation functions return a structured payload:

```typescript
{ 
  valid: boolean, 
  errors?: ValidationIssue[] 
}

```

This pattern allows both the document store and migration engine to consume validation results programmatically, aborting operations when structural checks fail.

### Validating Scenes and Stages

**`validateAppScene`** and **`validateAppStage`** serve as the primary entry points for write-time validation. When a user creates or updates a scene through the UI, the application invokes these functions to verify that the payload contains non-empty strings, numeric ordering, and matching `type`/`content.type` pairs. If validation fails, the write aborts before IndexedDB storage occurs, preventing corrupt data from entering the document store.

## Migration-Time Verification

OpenMAIC employs **lazy migration** to upgrade legacy documents on demand rather than batch-processing stored data. The migration engine runs the same Zod validators on both source analysis and destination output to guarantee conversion accuracy.

### The Three-Phase Migration Pipeline

The **`migrateLocked`** function in **[`lib/document-store/migration.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/document-store/migration.ts)** orchestrates document migration through three distinct phases, using validators at critical checkpoints:

1. **Probe Phase**: Reads the current destination document and legacy snapshot. If a destination exists, **`assertValidDestination`** invokes `validateAppStage` and `validateAppScene` to confirm the stored document already conforms to the current schema.
2. **Convert Phase**: Transforms legacy media references into allocated asset IDs without holding a lock. No validation occurs here because the conversion may produce intermediate structures.
3. **Commit Phase**: Re-acquires the lock, reconciles concurrent edits, and persists the converted document. After conversion, **`assertValidDestination`** runs again on the new document. If the migration produced a canonical document from the legacy snapshot, **`assertMigrationVerified`** re-runs validators to confirm the DSL migration (`migrate`) produced the same structural shape as the canonical snapshot.

### Migration Markers and Data Safety

Once validation succeeds, the migration machinery records a marker in IndexedDB using the key pattern `document-migration:<stageId>`. This prevents redundant migration work on subsequent accesses. If any validator reports an error during the pipeline, the migration aborts entirely, leaving the legacy document untouched to preserve data safety.

## Validation Strategy and Error Handling

The validators implement a deliberate balance between backward compatibility and structural rigor.

### Lenient on History, Strict on Structure

OpenMAIC's validation layer is **lenient on historical data**—allowing unknown action types to pass through—while remaining **strict on required fields** such as non-empty strings, numeric order fields, and type discriminator matching. This approach enables older documents to survive schema evolution without manual intervention, while still blocking payloads that would crash the playback engine due to missing critical fields.

### Runtime Integration

The **`accessDocument`** and **`mutateDocument`** functions exported from **[`lib/document-store/store.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/document-store/store.ts)** provide the high-level API for document interaction. These functions automatically invoke the migration pipeline when detecting legacy data, ensuring that callers always receive Zod-validated documents regardless of when the data was originally created.

## Practical Implementation Examples

### Validating a Scene Before Persistence

```typescript
import { validateAppScene } from '@/lib/document-store/validators';

const scene = {/* …scene data from UI… */};
const result = validateAppScene(scene);

if (!result.valid) {
  console.error('Scene validation failed', result.errors);
  // Reject the write or correct the payload
}

```

### Loading a Document with Automatic Migration

```typescript
import { accessDocument } from '@/lib/document-store/migration';

async function openStage(stageId: string) {
  const { document, readOnlyLegacy } = await accessDocument(stageId);
  
  if (document) {
    console.log('Document ready', document);
    // `document` has passed Zod validation already
  } else if (readOnlyLegacy) {
    console.warn('Stage only has legacy data – migration deferred');
  }
}

```

### Mutating a Document Safely

```typescript
import { mutateDocument } from '@/lib/document-store/migration';
import { validateAppScene } from '@/lib/document-store/validators';

async function addScene(stageId: string, newScene) {
  await mutateDocument(stageId, async (doc, store) => {
    const validated = validateAppScene(newScene);
    if (!validated.valid) throw new Error('Invalid scene');

    const updated = doc 
      ? { ...doc, scenes: [...doc.scenes, newScene] } 
      : { stage: {}, scenes: [newScene] };
    
    await store.saveDocument(updated);
  });
}

```

## Summary

- OpenMAIC validates documents at two critical points: **write-time** (via `validateAppScene` and `validateAppStage`) and **migration-time** (via `assertValidDestination` and `assertMigrationVerified`).
- The validation implementation in **[`lib/document-store/validators.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/document-store/validators.ts)** imports Zod primitives from `@openmaic/dsl` and extends them for application-specific scene unions.
- The **`migrateLocked`** routine in **[`lib/document-store/migration.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/document-store/migration.ts)** executes a three-phase pipeline (Probe, Convert, Commit) that validates both source and destination documents before persisting changes.
- Failed validation immediately aborts migration, preserving legacy data and recording migration markers (`document-migration:<stageId>`) only after successful verification.
- Validators tolerate unknown historical fields while enforcing strict requirements on critical structural fields, preventing runtime crashes while maintaining backward compatibility.

## Frequently Asked Questions

### What is the validation payload format returned by OpenMAIC validators?

OpenMAIC validators return an object with the shape `{ valid: boolean, errors?: ValidationIssue[] }`. The `valid` boolean indicates structural conformity, while the optional `errors` array contains detailed descriptions of validation failures that the migration engine or UI can consume for error reporting.

### How does OpenMAIC prevent data loss during document migration?

The migration pipeline in **[`lib/document-store/migration.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/document-store/migration.ts)** performs validation checks in the Probe and Commit phases using `assertValidDestination` and `assertMigrationVerified`. If Zod validation fails at any checkpoint, the operation aborts before overwriting legacy data. The system only records migration markers in IndexedDB after successful validation, ensuring that corrupt conversions never replace valid legacy snapshots.

### Where does the Zod schema originate in OpenMAIC?

The base Zod schemas and type guards—such as `isActionType`, `validateAction`, `validateScene`, and `validateStage`—originate from the **`@openmaic/dsl`** package. The **[`lib/document-store/validators.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/document-store/validators.ts)** file imports these primitives and composes them into higher-level functions like `validateAppScene` that handle the application's extended scene types (interactive, PBL, etc.).

### How does lazy migration work in OpenMAIC?

OpenMAIC implements lazy migration through the **`accessDocument`** function, which checks for legacy data when loading a stage. If legacy data exists, the system invokes **`migrateLocked`** to convert the document on demand rather than batch-processing stored documents. The conversion occurs in three phases (Probe, Convert, Commit), with Zod validators verifying the output before the function returns the document to the caller, ensuring that the application always works with current-schema data.