# How the Playback Engine Manages the Classroom Lifecycle in OpenMAIC

> Discover how the OpenMAIC Playback Engine manages the classroom lifecycle using a state machine. Learn about transitions between idle, playing, paused, and live modes for seamless lecture and discussion management.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: internals
- Published: 2026-08-30

---

**The Playback Engine in OpenMAIC orchestrates the classroom lifecycle through a finite state machine with four distinct modes—idle, playing, paused, and live—transitioning between lecture playback and interactive discussions via methods like `start()`, `handleUserInterrupt()`, and `confirmDiscussion()`.**

OpenMAIC’s **Playback Engine** is the core state-machine that governs every phase of a classroom session, from initial lecture playback through live student discussions to final cleanup. Unlike complex orchestration layers that compile intermediate representations, this engine directly consumes the `Scene.actions[]` array and reacts immediately to user-initiated events. Understanding how this engine manages the classroom lifecycle is essential for developers extending OpenMAIC’s educational platform or debugging session continuity issues.

## Core State Machine Architecture

The engine operates through four mutually exclusive modes defined in [`lib/playback/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/types.ts). Each mode represents a distinct phase of the classroom lifecycle, with strict transitions ensuring predictable behavior during lecture playback and discussion interruptions.

### The Four Engine Modes

- **idle** — No playback activity. The engine awaits a new lecture start, holding no active timers or audio resources.
- **playing** — Auto-advancing through speech, spotlights, whiteboard actions, and other scripted scene elements.
- **paused** — Temporary halt state preserving playback position, used when users press pause or when ProactiveCards appear.
- **live** — Interactive discussion state with an active SSE stream processing user chat messages in real-time.

### State Transition Entry and Exit Points

The state diagram sketched in [`lib/playback/engine.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/engine.ts)【/cache/repos/github.com/THU-MAIC/OpenMAIC/main/lib/playback/engine.ts#L7-L24】 defines specific methods that trigger transitions:

- `idle` → `playing` occurs via `start()` or `continuePlayback()`
- `playing` → `paused` occurs via `pause()`
- `playing` → `live` occurs via `handleUserInterrupt()` (user chat) or `confirmDiscussion()` (ProactiveCard join)
- `live` → `idle` occurs via `handleEndDiscussion()` or `handleDiscussionError()`
- Any mode → `idle` can be forced via `stop()`

## Key Lifecycle Hooks

The engine exposes explicit methods for each lifecycle phase, allowing the UI layer to control session flow while the engine manages internal state consistency.

### Starting and Resuming Sessions

The `start()` method resets indices, clears stale generation caches, switches the mode to **playing**, and immediately invokes `processNext()` to begin action execution. For resuming after a discussion, `continuePlayback()` preserves the current cursor position rather than resetting it, ensuring seamless return to lecture content.

```typescript
import { PlaybackEngine } from '@/lib/playback/engine';
import { ActionEngine } from '@/lib/action/engine';
import { AudioPlayer } from '@/lib/utils/audio-player';

const scenes = loadLessonScenes();
const engine = new PlaybackEngine(
  scenes,
  new ActionEngine(),
  new AudioPlayer(),
  {
    onModeChange: (mode) => console.log('Engine mode →', mode),
    onProgress: (snapshot) => saveSnapshot(snapshot),
    onProactiveShow: (trigger) => showProactiveCard(trigger),
    onDiscussionConfirmed: (q, p, a) => startLiveDiscussion(q, p, a),
  },
);

// Initiate classroom session
engine.start();

```

### Handling User Interruptions

When a student sends a chat message during playback, `handleUserInterrupt(text)` saves the current lecture cursor, stops ongoing audio/TTS, and transitions the engine to **live** mode. This method is the primary entry point for converting passive lecture viewing into active discussion.

```typescript
function sendMessage(text: string) {
  // Engine saves cursor and switches to live mode
  playbackEngine.handleUserInterrupt(text);
  sendChatToBackend(text);
}

```

### Managing Discussion Flow

The engine provides three distinct methods for handling ProactiveCard interactions:

- **`confirmDiscussion()`** — Consumes the discussion, saves the lecture cursor to `savedSceneIndex`/`ActionIndex`, switches to **live** mode, and notifies UI callbacks. Called when users click "Join" on a discussion prompt.
- **`skipDiscussion()`** — Marks the discussion as consumed without entering **live** mode, immediately continuing playback.
- **`handleEndDiscussion()`** — Restores the saved lecture position, closes the whiteboard, fires `onDiscussionEnd`, and returns to **idle** when users finish a discussion.

```typescript
function onJoinDiscussion() {
  playbackEngine.confirmDiscussion(); // Moves to live mode
}

function onSkipDiscussion() {
  playbackEngine.skipDiscussion(); // Continues lecture
}

```

### Emergency Stopping and Cleanup

The `stop()` method forces the engine to **idle**, halting all audio/TTS, clearing effects, and resetting indices. This is invoked when users navigate away from the classroom or when the application tears down.

## Integration with the Classroom UI

The engine exposes callbacks (`onModeChange`, `onProgress`, `onProactiveShow`) that allow the workbench UI layer to react instantly to state changes without polling.

### Pro Playback Exit Handling

When users exit full-screen Pro playback mode, `exitProPlaybackToStandalone` in [`lib/workbench/pro-playback-exit.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/workbench/pro-playback-exit.ts)【/cache/repos/github.com/THU-MAIC/OpenMAIC/main/lib/workbench/pro-playback-exit.ts#L19-L34】 tears down the playback context and ensures the classroom UI re-mounts with a clean state.

```typescript
import { exitProPlaybackToStandalone } from '@/lib/workbench/pro-playback-exit';

function leaveProPlayback(stageId: string) {
  exitProPlaybackToStandalone({
    stageId,
    teardown: () => playbackEngine.stop(),
    setMode: (mode) => playbackEngine.setMode(mode as any),
    replace: (url) => router.replace(url),
  });
}

```

### Header Controls and Discussion UI

The classroom header controls in [`lib/workbench/classroom-header-controls.tsx`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/workbench/classroom-header-controls.tsx) invoke `engine.pause()` and `engine.resume()` based on play/pause button interactions. Meanwhile, the ProactiveCard component drives the discussion lifecycle by calling `engine.confirmDiscussion()` or `engine.skipDiscussion()`, creating a seamless "lecture → discussion → lecture" flow without position loss or visual flashes.

## Persistence and Recovery

The engine supports classroom lifecycle persistence through snapshots. The `getSnapshot()` method exports the current `sceneIndex`, `actionIndex`, and consumed discussion IDs, which the persistence layer stores in IndexedDB. After page reloads or navigation, `restoreFromSnapshot()` reinjects these indices and the consumed set, allowing the engine to resume exactly where the student left off.

## Timing and TTS Management

During the **playing** and **paused** modes, the engine manages audio timing through two mechanisms:

- **Pre-generated audio** uses `audioPlayer.play()` with exact duration tracking.
- **Browser TTS** uses the Web Speech API with `estimateSpeechDurationMs` as a fallback timer.

When `pause()` fires, the engine saves `speechTimerRemaining` and, for browser TTS, stores unfinished chunks in `browserTTSPausedChunks`. The `resume()` method restores these timers and re-schedules speech, ensuring that pausing never skips content regardless of audio source.

## Summary

- The Playback Engine manages the OpenMAIC classroom lifecycle through four strict modes: **idle**, **playing**, **paused**, and **live**.
- State transitions are triggered by specific methods—`start()`, `handleUserInterrupt()`, `confirmDiscussion()`, and `handleEndDiscussion()`—that preserve lecture position and manage audio resources.
- The engine directly consumes `Scene.actions[]` without intermediate compilation, making it lightweight and predictable.
- Persistence is achieved through `getSnapshot()` and `restoreFromSnapshot()`, enabling session recovery across page reloads.
- UI integration occurs through callbacks that update [`classroom-header-controls.tsx`](https://github.com/THU-MAIC/OpenMAIC/blob/main/classroom-header-controls.tsx) and handle Pro playback exits via [`pro-playback-exit.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/pro-playback-exit.ts).

## Frequently Asked Questions

### What triggers the transition from lecture playback to live discussion mode?

The engine transitions from **playing** to **live** mode via two paths: `handleUserInterrupt(text)` when a user sends a chat message during a lecture, or `confirmDiscussion()` when the user clicks "Join" on a ProactiveCard. Both methods save the current lecture cursor position before switching modes, ensuring the lecture can resume at the exact action where it was interrupted.

### How does the Playback Engine handle classroom session recovery after a browser refresh?

The engine exports a snapshot containing `sceneIndex`, `actionIndex`, and consumed discussion IDs via `getSnapshot()`. The application layer stores this in IndexedDB, and upon reload calls `restoreFromSnapshot()` to reinject these values. This mechanism allows the engine to resume the classroom lifecycle at the precise action and mode state that existed before the refresh.

### What is the difference between `stop()` and `pause()` in the Playback Engine?

`pause()` transitions the engine to **paused** mode while preserving the current `sceneIndex` and `actionIndex`, saving remaining speech timers and TTS chunks for later resumption. `stop()` forces an immediate return to **idle**, clearing all timers, resetting indices, and halting audio without preserving playback position—effectively ending the current classroom session.

### How does the engine manage audio timing when users pause a TTS lecture?

When `pause()` executes during browser TTS playback, the engine stores the remaining duration in `speechTimerRemaining` and saves unfinished utterance chunks in `browserTTSPausedChunks`. Upon `resume()`, it restores these values and re-schedules the speech timer, ensuring that pausing a lecture never results in skipped content or lost audio synchronization.