# How the Playback State Machine Works in OpenMAIC

> Explore the OpenMAIC playback state machine. Learn how it manages mode transitions, persistent cursors, and UI derivation for classroom session orchestration. Discover its core functionalities in lib/playback.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: internals
- Published: 2026-08-30

---

**OpenMAIC orchestrates recorded classroom sessions using a deterministic playback state machine implemented in `lib/playback` that coordinates mode transitions, persistent cursors, and reactive UI derivation across edit, playback, and autonomous modes.**

The OpenMAIC platform manages recorded educational content through a sophisticated playback state machine located in the `lib/playback` package. This system maintains synchronization across multiple clients while handling complex transitions between editing and playback modes. By separating action processing from UI state derivation, the architecture ensures consistent behavior whether users are scrubbing through recordings or resuming sessions on new devices.

## Core Architecture Components

The playback state machine centers on immutable snapshots, persistent cursors, and pure state derivation functions that together enable deterministic replay across distributed clients.

### PlaybackEngine

The `PlaybackEngine` class in [`lib/playback/engine.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/engine.ts) serves as the central orchestrator. It consumes discrete **actions**—such as `Play`, `Pause`, and `Seek`—and updates the current `PlaybackSnapshot`. The engine owns the **cursor** reference and exposes a `dispatch()` method for state mutations. According to the OpenMAIC source code, this class initializes automatically when the stage store loads via [`lib/playback/index.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/index.ts).

### PlaybackSnapshot and Types

Defined in [`lib/playback/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/types.ts), the `PlaybackSnapshot` interface represents an immutable view of playback progress. It stores the current position, playback speed, and boolean flags including `paused` and `ended`. These snapshots ensure predictable state transitions by preventing direct mutation of playback internals, allowing the engine to compute derived UI states deterministically.

### Cursor Persistence

The `Cursor` module in [`lib/playback/cursor.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/cursor.ts) provides device-scoped storage for resume positions. It persists the last played action ID per device using `cursor.set(actionId)` after each action completes, enabling `cursor.get()` to retrieve the exact position when clients reconnect. The system automatically clears this storage on stage deletion, as verified in [`stage-delete-wiring.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/stage-delete-wiring.test.ts).

### Resume Action Resolution

The [`lib/playback/action-resume.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/action-resume.ts) module encapsulates decision logic for session resumption. It evaluates whether to dispatch a simple `Play` action or a compound `Seek` followed by `Play`, depending on the current mode and the presence of a valid cursor position. This abstraction ensures consistent resume semantics across different application entry points.

## State Transition Flow

The playback state machine follows a strict lifecycle from initialization through auto-resume, with clear transitions between the three global modes: `edit`, `playback`, and `autonomous`.

### Initialization and Mode Entry

When a classroom session initializes, the application creates a `PlaybackEngine` instance exported from [`lib/playback/index.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/index.ts). The engine begins in **edit mode** with `paused: true`. To enter playback, the UI dispatches a `SET_PLAYBACK_ON` flag (validated in [`workspace-panes.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/workspace-panes.test.ts)), transitioning the stage store's `mode` property to `'playback'`. The engine then receives a `Play` action, setting `paused: false` and beginning time updates.

### Navigation and Seeking

User navigation commands convert to discrete actions through [`lib/playback/action-navigation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/action-navigation.ts). For example, clicking "skip forward" invokes `seekToAction()`, which constructs a `Seek` action targeting a specific action ID. The engine processes this by updating the snapshot and persisting the new position via the cursor.

### Auto-Resume on Client Join

When a new client connects to an active session, [`lib/playback/auto-resume.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/auto-resume.ts) checks if `stageStore.mode === 'playback'`. If true, it reads the persisted cursor position using `cursor.get()`, dispatches a `Seek` action to that position, then issues a `Play` command. This ensures seamless continuity across device switches without manual seeking.

## Derived State Management

UI components do not query the engine directly. Instead, they consume **derived state** from [`lib/playback/derived-state.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/derived-state.ts). Pure functions compute boolean flags such as `isPlayback`, `showSpotlight`, and `showSpeedControl` from the current `PlaybackSnapshot`. These values are memoized and recalculated only when the snapshot changes, preventing unnecessary re-renders while ensuring the playback chrome, speed controls, and spotlight appear only in appropriate contexts.

## Edge Cases and Resilience

The state machine handles several edge cases through defensive checks and comprehensive test coverage. The system ignores stale playback flags when no course is open, preventing phantom UI elements as tested in [`workspace-course-chat-bootstrap.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/workspace-course-chat-bootstrap.test.ts). Pane locking logic ensures that locked editing panes never resolve to playback mode even when the stage store indicates `playback`, as asserted in [`classroom-pane-edit-lock.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/classroom-pane-edit-lock.test.ts). Audio scaling respects `playbackSpeed` adjustments, verified in [`video-export/timeline.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/video-export/timeline.test.ts), while the renderer selection logic handles legacy versus new canvas implementations as checked in [`slide-renderer/screen-canvas-renderer.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/slide-renderer/screen-canvas-renderer.test.ts).

## Practical Implementation Examples

### Initializing the Playback Engine

```typescript
import { PlaybackEngine } from '@/lib/playback/engine';
import { createStageStore } from '@/store/stage';

const stageStore = createStageStore();
const playback = new PlaybackEngine(stageStore);

// Start playback
playback.dispatch({ type: 'Play' });

```

### Seeking to Specific Actions

```typescript
import { seekToAction } from '@/lib/playback/action-navigation';

// Jump to action ID 'a42'
playback.dispatch(seekToAction('a42'));

```

### Implementing Auto-Resume

```typescript
import { autoResume } from '@/lib/playback/auto-resume';

if (stageStore.mode === 'playback') {
  await autoResume(playback); // Reads cursor, seeks, plays
}

```

### React Component Integration

```tsx
import { useDerivedPlaybackState } from '@/lib/playback/derived-state';

export default function PlaybackChrome() {
  const { isPlayback, showSpeedControl, showSpotlight } = useDerivedPlaybackState();

  if (!isPlayback) return null;

  return (
    <div className="playback-chrome">
      {showSpeedControl && <SpeedControl />}
      {showSpotlight && <Spotlight />}
    </div>
  );
}

```

## Summary

- The **PlaybackEngine** in [`lib/playback/engine.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/engine.ts) processes immutable actions and owns the playback cursor referenced by [`cursor.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/cursor.ts).
- **PlaybackSnapshot** from [`lib/playback/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/types.ts) provides thread-safe state representation containing `paused`, `ended`, and speed properties.
- The **Cursor** module enables cross-device resume through `cursor.set()` and `cursor.get()` operations, cleared automatically on stage deletion.
- **Derived state** functions in [`lib/playback/derived-state.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/derived-state.ts) compute UI visibility flags such as `showSpotlight` and `showSpeedControl` from snapshots.
- **Auto-resume** logic in [`lib/playback/auto-resume.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/auto-resume.ts) coordinates with [`lib/playback/action-resume.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/action-resume.ts) to restore playback position for joining clients.
- Comprehensive tests in [`stage-delete-wiring.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/stage-delete-wiring.test.ts), [`workspace-panes.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/workspace-panes.test.ts), and [`classroom-pane-edit-lock.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/classroom-pane-edit-lock.test.ts) verify cursor cleanup and mode gating.

## Frequently Asked Questions

### How does the PlaybackEngine handle mode transitions between edit and playback?

The `PlaybackEngine` does not store mode directly; instead, it reacts to the stage store's `mode` property. When the UI dispatches `SET_PLAYBACK_ON`, the stage store transitions to `'playback'` mode and the engine receives a `Play` action. Conversely, toggling playback off dispatches `Pause` and returns the store to `'edit'` mode. This separation ensures the engine remains focused on temporal state while the store manages global modality.

### What determines whether a client auto-resumes or starts from the beginning?

The `autoResume` function in [`lib/playback/auto-resume.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/auto-resume.ts) checks two conditions: whether `stageStore.mode === 'playback'` and whether a cursor exists via `cursor.get()`. If both conditions are met, it dispatches a `Seek` to the stored action ID followed by `Play`. If no cursor exists or the mode is not playback, the client initializes without seeking, effectively starting from the beginning.

### How does the system prevent playback UI from appearing in inappropriate contexts?

The **derived state** system in [`lib/playback/derived-state.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/derived-state.ts) computes `isPlayback` and related flags by combining the snapshot state with external conditions such as pane locking and course availability. Tests in [`workspace-panes.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/workspace-panes.test.ts) and [`classroom-pane-edit-lock.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/classroom-pane-edit-lock.test.ts) verify that these pure functions return false when panes are locked or no course is active, ensuring the playback chrome remains hidden.

### Where is the cursor position persisted, and when is it cleared?

The cursor persists per-device to local storage through the `Cursor` class in [`lib/playback/cursor.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/cursor.ts), updated via `cursor.set(actionId)` after each playback action. The system clears this data when the stage deletes, as implemented in the wiring tested by [`stage-delete-wiring.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/stage-delete-wiring.test.ts), preventing stale resume positions from accumulating across session boundaries.