How the Playback Engine Manages the Classroom Lifecycle in OpenMAIC

The Playback Engine in OpenMAIC orchestrates the classroom lifecycle through a finite state machine with four distinct modes—idle, playing, paused, and live—transitioning between lecture playback and interactive discussions via methods like start(), handleUserInterrupt(), and confirmDiscussion().

OpenMAIC’s Playback Engine is the core state-machine that governs every phase of a classroom session, from initial lecture playback through live student discussions to final cleanup. Unlike complex orchestration layers that compile intermediate representations, this engine directly consumes the Scene.actions[] array and reacts immediately to user-initiated events. Understanding how this engine manages the classroom lifecycle is essential for developers extending OpenMAIC’s educational platform or debugging session continuity issues.

Core State Machine Architecture

The engine operates through four mutually exclusive modes defined in lib/playback/types.ts. Each mode represents a distinct phase of the classroom lifecycle, with strict transitions ensuring predictable behavior during lecture playback and discussion interruptions.

The Four Engine Modes

• idle — No playback activity. The engine awaits a new lecture start, holding no active timers or audio resources.
• playing — Auto-advancing through speech, spotlights, whiteboard actions, and other scripted scene elements.
• paused — Temporary halt state preserving playback position, used when users press pause or when ProactiveCards appear.
• live — Interactive discussion state with an active SSE stream processing user chat messages in real-time.

State Transition Entry and Exit Points

The state diagram sketched in lib/playback/engine.ts【/cache/repos/github.com/THU-MAIC/OpenMAIC/main/lib/playback/engine.ts#L7-L24】 defines specific methods that trigger transitions:

• idle → playing occurs via start() or continuePlayback()
• playing → paused occurs via pause()
• playing → live occurs via handleUserInterrupt() (user chat) or confirmDiscussion() (ProactiveCard join)
• live → idle occurs via handleEndDiscussion() or handleDiscussionError()
• Any mode → idle can be forced via stop()

Key Lifecycle Hooks

The engine exposes explicit methods for each lifecycle phase, allowing the UI layer to control session flow while the engine manages internal state consistency.

Starting and Resuming Sessions

The start() method resets indices, clears stale generation caches, switches the mode to playing, and immediately invokes processNext() to begin action execution. For resuming after a discussion, continuePlayback() preserves the current cursor position rather than resetting it, ensuring seamless return to lecture content.

import { PlaybackEngine } from '@/lib/playback/engine';
import { ActionEngine } from '@/lib/action/engine';
import { AudioPlayer } from '@/lib/utils/audio-player';

const scenes = loadLessonScenes();
const engine = new PlaybackEngine(
  scenes,
  new ActionEngine(),
  new AudioPlayer(),
  {
    onModeChange: (mode) => console.log('Engine mode →', mode),
    onProgress: (snapshot) => saveSnapshot(snapshot),
    onProactiveShow: (trigger) => showProactiveCard(trigger),
    onDiscussionConfirmed: (q, p, a) => startLiveDiscussion(q, p, a),
  },
);

// Initiate classroom session
engine.start();

Handling User Interruptions

When a student sends a chat message during playback, handleUserInterrupt(text) saves the current lecture cursor, stops ongoing audio/TTS, and transitions the engine to live mode. This method is the primary entry point for converting passive lecture viewing into active discussion.

function sendMessage(text: string) {
  // Engine saves cursor and switches to live mode
  playbackEngine.handleUserInterrupt(text);
  sendChatToBackend(text);
}

Managing Discussion Flow

The engine provides three distinct methods for handling ProactiveCard interactions:

• confirmDiscussion() — Consumes the discussion, saves the lecture cursor to savedSceneIndex/ActionIndex, switches to live mode, and notifies UI callbacks. Called when users click "Join" on a discussion prompt.
• skipDiscussion() — Marks the discussion as consumed without entering live mode, immediately continuing playback.
• handleEndDiscussion() — Restores the saved lecture position, closes the whiteboard, fires onDiscussionEnd, and returns to idle when users finish a discussion.

function onJoinDiscussion() {
  playbackEngine.confirmDiscussion(); // Moves to live mode
}

function onSkipDiscussion() {
  playbackEngine.skipDiscussion(); // Continues lecture
}

Emergency Stopping and Cleanup

The stop() method forces the engine to idle, halting all audio/TTS, clearing effects, and resetting indices. This is invoked when users navigate away from the classroom or when the application tears down.

Integration with the Classroom UI

The engine exposes callbacks (onModeChange, onProgress, onProactiveShow) that allow the workbench UI layer to react instantly to state changes without polling.

Pro Playback Exit Handling

When users exit full-screen Pro playback mode, exitProPlaybackToStandalone in lib/workbench/pro-playback-exit.ts【/cache/repos/github.com/THU-MAIC/OpenMAIC/main/lib/workbench/pro-playback-exit.ts#L19-L34】 tears down the playback context and ensures the classroom UI re-mounts with a clean state.

import { exitProPlaybackToStandalone } from '@/lib/workbench/pro-playback-exit';

function leaveProPlayback(stageId: string) {
  exitProPlaybackToStandalone({
    stageId,
    teardown: () => playbackEngine.stop(),
    setMode: (mode) => playbackEngine.setMode(mode as any),
    replace: (url) => router.replace(url),
  });
}

Header Controls and Discussion UI

The classroom header controls in lib/workbench/classroom-header-controls.tsx invoke engine.pause() and engine.resume() based on play/pause button interactions. Meanwhile, the ProactiveCard component drives the discussion lifecycle by calling engine.confirmDiscussion() or engine.skipDiscussion(), creating a seamless "lecture → discussion → lecture" flow without position loss or visual flashes.

Persistence and Recovery

The engine supports classroom lifecycle persistence through snapshots. The getSnapshot() method exports the current sceneIndex, actionIndex, and consumed discussion IDs, which the persistence layer stores in IndexedDB. After page reloads or navigation, restoreFromSnapshot() reinjects these indices and the consumed set, allowing the engine to resume exactly where the student left off.

Timing and TTS Management

During the playing and paused modes, the engine manages audio timing through two mechanisms:

• Pre-generated audio uses audioPlayer.play() with exact duration tracking.
• Browser TTS uses the Web Speech API with estimateSpeechDurationMs as a fallback timer.

When pause() fires, the engine saves speechTimerRemaining and, for browser TTS, stores unfinished chunks in browserTTSPausedChunks. The resume() method restores these timers and re-schedules speech, ensuring that pausing never skips content regardless of audio source.

Summary

• The Playback Engine manages the OpenMAIC classroom lifecycle through four strict modes: idle, playing, paused, and live.
• State transitions are triggered by specific methods—start(), handleUserInterrupt(), confirmDiscussion(), and handleEndDiscussion()—that preserve lecture position and manage audio resources.
• The engine directly consumes Scene.actions[] without intermediate compilation, making it lightweight and predictable.
• Persistence is achieved through getSnapshot() and restoreFromSnapshot(), enabling session recovery across page reloads.
• UI integration occurs through callbacks that update classroom-header-controls.tsx and handle Pro playback exits via pro-playback-exit.ts.

Frequently Asked Questions

What triggers the transition from lecture playback to live discussion mode?

The engine transitions from playing to live mode via two paths: handleUserInterrupt(text) when a user sends a chat message during a lecture, or confirmDiscussion() when the user clicks "Join" on a ProactiveCard. Both methods save the current lecture cursor position before switching modes, ensuring the lecture can resume at the exact action where it was interrupted.

How does the Playback Engine handle classroom session recovery after a browser refresh?

The engine exports a snapshot containing sceneIndex, actionIndex, and consumed discussion IDs via getSnapshot(). The application layer stores this in IndexedDB, and upon reload calls restoreFromSnapshot() to reinject these values. This mechanism allows the engine to resume the classroom lifecycle at the precise action and mode state that existed before the refresh.

What is the difference between stop() and pause() in the Playback Engine?

pause() transitions the engine to paused mode while preserving the current sceneIndex and actionIndex, saving remaining speech timers and TTS chunks for later resumption. stop() forces an immediate return to idle, clearing all timers, resetting indices, and halting audio without preserving playback position—effectively ending the current classroom session.

How does the engine manage audio timing when users pause a TTS lecture?

When pause() executes during browser TTS playback, the engine stores the remaining duration in speechTimerRemaining and saves unfinished utterance chunks in browserTTSPausedChunks. Upon resume(), it restores these values and re-schedules the speech timer, ensuring that pausing a lecture never results in skipped content or lost audio synchronization.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}