# How OpenMAIC Records and Replays Whiteboard Actions for Cross-Turn Consistency

> OpenMAIC records whiteboard actions as immutable objects with sequence numbers, storing and replaying them to ensure consistent visual states across agent turns. Learn how this ensures cross-turn consistency.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: internals
- Published: 2026-09-04

---

**OpenMAIC captures every whiteboard modification as an immutable action object with a monotonic sequence number, stores these actions in an ordered log, and replays them sequentially at each turn boundary to reconstruct an identical visual state for every agent.**

The **THU-MAIC/OpenMAIC** repository implements a deterministic whiteboard synchronization mechanism that ensures agents evaluate the exact same layout during every turn. By treating whiteboard interactions as a stream of replayable commands rather than mutable state snapshots, the system eliminates visual drift caused by concurrent modifications or differing rendering pipelines.

## The Action Recording Pipeline

When users or agents modify the whiteboard canvas, the system does not immediately mutate a shared model. Instead, it generates lightweight descriptors that can be deterministically replayed.

### Action Descriptor Structure

In [`eval/whiteboard-layout/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/eval/whiteboard-layout/types.ts), each interaction is codified as a **WhiteboardAction** containing three critical fields:

- **`seq`** – A monotonically increasing integer that preserves strict temporal ordering
- **`type`** – The operation category (`'add'`, `'move'`, `'delete'`, or `'draw'`)
- **`payload`** – Minimal data required to reproduce the change (element IDs, coordinates, content strings, or style objects)

This structure decouples the user interface events from the internal state representation, allowing actions to be serialized, transmitted over network boundaries, and replayed on any compatible runtime.

### Recording Logic in [`state-manager.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/state-manager.ts)

The `recordAction` function in [`eval/whiteboard-layout/state-manager.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/eval/whiteboard-layout/state-manager.ts) handles the append-only logic:

```typescript
export interface WhiteboardAction {
  seq: number;
  type: 'add' | 'move' | 'delete' | 'draw';
  payload: Record<string, any>;
}

export function recordAction(
  state: WhiteboardState, 
  action: Omit<WhiteboardAction, 'seq'>
): void {
  const seq = state.nextSeq++;
  const fullAction: WhiteboardAction = { seq, ...action };
  state.log.push(fullAction);  // Append to ordered log
}

```

The function assigns the next available sequence number and pushes the complete descriptor into `state.log`, an array that serves as the single source of truth for the current turn's modifications.

## Persisting Whiteboard State Between Turns

To maintain consistency when transitioning between agent turns, the action log undergoes serialization and hydration cycles. The **state-manager.ts** module exposes two utilities for this purpose:

```typescript
export function serializeLog(state: WhiteboardState): string {
  return JSON.stringify(state.log);
}

export function hydrateLog(serialized: string): WhiteboardAction[] {
  return JSON.parse(serialized) as WhiteboardAction[];
}

```

When an agent completes its turn, `serializeLog` converts the accumulated action array into a JSON string that can be stored in session state or transmitted via API. The subsequent turn invokes `hydrateLog` to receive an identical copy of the action history, ensuring the next agent begins with the exact same operation sequence as its predecessor.

## Replay Engine for Deterministic Reconstruction

The critical consistency guarantee comes from the replay mechanism, which rebuilds the whiteboard model from scratch using only the action log, eliminating dependency on mutable shared memory.

### Strict Sequence Ordering in `replayLog`

The `replayLog` function, implemented in [`eval/whiteboard-layout/scorer.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/eval/whiteboard-layout/scorer.ts), processes actions in sequence order to reconstruct the visual state:

```typescript
export function replayLog(actions: WhiteboardAction[]): WhiteboardModel {
  const model: WhiteboardModel = { elements: {} };

  actions
    .sort((a, b) => a.seq - b.seq)  // Enforce strict ordering
    .forEach(action => {
      switch (action.type) {
        case 'add':
          model.elements[action.payload.id] = { ...action.payload };
          break;
        case 'move':
          const el = model.elements[action.payload.id];
          if (el) { 
            el.x = action.payload.x; 
            el.y = action.payload.y; 
          }
          break;
        case 'delete':
          delete model.elements[action.payload.id];
          break;
        case 'draw':
          model.elements[action.payload.id].path.push(action.payload.point);
          break;
      }
    });

  return model;
}

```

By sorting on `seq` before execution, the engine guarantees that concurrent or out-of-order deliveries do not corrupt the final state. Each handler operates as a pure function, mutating only the local `model` object without side effects on external systems.

### Integration with Scoring Logic

After reconstruction, the resulting `WhiteboardModel` feeds directly into the evaluation logic within [`eval/whiteboard-layout/scorer.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/eval/whiteboard-layout/scorer.ts). This ensures that scoring algorithms analyze the precise visual layout that human users observe, because both views derive from the identical action log processed through the same deterministic replay pipeline.

## Summary

- **Immutable Actions**: Every canvas interaction generates an immutable descriptor containing a sequence number, operation type, and payload.
- **Ordered Logging**: The `recordAction` function in [`eval/whiteboard-layout/state-manager.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/eval/whiteboard-layout/state-manager.ts) maintains an append-only log that preserves strict temporal ordering via the `seq` field.
- **JSON Serialization**: Action logs serialize to JSON for persistence across turn boundaries using `serializeLog` and `hydrateLog`.
- **Deterministic Replay**: The `replayLog` function reconstructs the whiteboard model from scratch by processing actions in sequence order, ensuring identical state across all agent instances.
- **Pure Functional Design**: The entire pipeline avoids side effects by rebuilding state rather than mutating shared references, preventing drift between user and agent views.

## Frequently Asked Questions

### What data structure does OpenMAIC use to store whiteboard actions?

OpenMAIC stores whiteboard actions as an array of **WhiteboardAction** objects, where each object implements the interface defined in [`eval/whiteboard-layout/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/eval/whiteboard-layout/types.ts). The structure includes a `seq` number for ordering, a `type` string categorizing the operation, and a `payload` object containing operation-specific data. This array lives within the `WhiteboardState` object managed by [`eval/whiteboard-layout/state-manager.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/eval/whiteboard-layout/state-manager.ts).

### How does OpenMAIC ensure actions are replayed in the correct order?

The system enforces ordering through the **monotonic `seq` property** assigned during recording. When replaying, the `replayLog` function in [`eval/whiteboard-layout/scorer.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/eval/whiteboard-layout/scorer.ts) explicitly sorts the action array using `(a, b) => a.seq - b.seq` before processing, ensuring that moves, deletions, and additions execute in the exact sequence they originally occurred.

### Why does OpenMAIC rebuild the whiteboard model from scratch each turn?

OpenMAIC employs **pure functional reconstruction** rather than incremental updates to eliminate state drift. By passing the action log through `replayLog` to generate a fresh `WhiteboardModel` at every turn boundary, the system guarantees that all agents—regardless of when they joined the session or how many UI events occurred—compute identical internal representations from the same canonical history.

### Where is the whiteboard action log maintained in the codebase?

The action log is maintained primarily in **[`eval/whiteboard-layout/state-manager.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/eval/whiteboard-layout/state-manager.ts)**, which provides the `recordAction`, `serializeLog`, and `hydrateLog` functions. The replay and scoring logic resides in **[`eval/whiteboard-layout/scorer.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/eval/whiteboard-layout/scorer.ts)**, while the TypeScript interfaces defining the action shapes are declared in **[`eval/whiteboard-layout/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/eval/whiteboard-layout/types.ts)**.