Classroom Playback State Machine in OpenMAIC: How `lib/playback` Manages Live Interaction

The OpenMAIC classroom playback system uses a deterministic PlaybackEngine state machine with four engine modes (idle, playing, paused, live) and three topic states to coordinate seamless transitions between lecture playback and live discussions.

The OpenMAIC project implements a sophisticated state management layer in lib/playback that governs how AI-powered classroom sessions alternate between scripted content and real-time student interaction. This article examines the PlaybackEngine implementation, its state transitions, and the callback-driven architecture that keeps the UI synchronized with engine state.

Core Engine Modes and State Machine Design

The PlaybackEngine class implements a compact state machine defined in [lib/playback/engine.ts](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/engine.ts) (lines 7–23). The engine operates in one of four engine modes, represented by the EngineMode type declared in [lib/playback/types.ts](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/types.ts) at line 18:

Mode	Purpose
idle	Initial state—no playback active, no discussion in progress
playing	Lecture playback running with audio, visuals, and timed actions
paused	Playback halted, timers and TTS suspended
live	Live discussion active—student chat or proactive Q&A session

The state machine diagram embedded in the source code shows how these modes interconnect:

idle ──► playing ──► paused
  ▲          ▲          │
  │          │          │
  │          │ resume() │
  │          └──────────┘
  │
  │  handleEndDiscussion()
  │  confirmDiscussion()
  │  / handleUserInterrupt()
  │        │
  │        ▼   pause()
  └─────── live ──► paused
          ▲        │
          │ resume / user msg
          └────────┘

State Transitions and Trigger Methods

All state transitions are deterministic and invoked through explicit methods on PlaybackEngine. Here are the critical transition paths:

Standard Playback Control

• start() — idle → playing: Initializes playback from the first scene, begins audio and action scheduling.

• pause() — playing → paused: Halts timers, pauses browser-native TTS, freezes the lecture cursor.

• resume() — paused → playing: Restores timers and audio, continues from the saved cursor position.

Live Discussion Integration

• confirmDiscussion() — playing → live: Triggered when the user accepts a proactive Q&A prompt. Saves the current lecture cursor, consumes the discussion trigger, and switches to live mode.

• skipDiscussion() — playing → playing: Discards a pending discussion trigger and proceeds with lecture playback.

• handleEndDiscussion() — live → idle: Closes the active discussion, restores the saved lecture state, and returns to idle (ready for resume() to restart playback).

• handleUserInterrupt() — playing/paused → live: Captures an unsolicited student message, immediately saves the interrupted cursor position, aborts playback, and opens live chat.

Error Handling and Cleanup

• handleDiscussionError() — * → idle: Emergency reset when a live discussion fails (network error, agent unavailability).

• stop() — * → idle: Nuclear option—cancels all work, clears state, returns to initial idle condition.

Topic State Tracking

Separate from engine mode, the system tracks topic state for live discussions using the TopicState type defined in [types.ts](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/types.ts) at line 21:

State	Meaning
active	Discussion currently open and interactive
pending	Discussion paused but not closed—can be resumed
closed	Discussion terminated, no further interaction possible

This distinction enables sophisticated behaviors: when pause() is called during live mode, the topic becomes pending rather than closing entirely. Subsequent resume() calls detect this state and return to live rather than playing.

Callback Architecture

The PlaybackEngine accepts a PlaybackEngineCallbacks object that decouples state logic from UI rendering. Key callbacks include:

• onModeChange(mode: EngineMode) — Fires on every state transition
• onProactiveShow(trigger) — Signals a discussion opportunity is available
• onDiscussionConfirmed(topic, prompt, agentId) — Confirms user entered live mode
• onUserInterrupt() — Indicates spontaneous chat initiation
• onProgress(snapshot) — Delivers playback progress for progress bars

This design enables deterministic unit testing and clean separation between engine logic and React/Vue components.

Practical Code Examples

Basic Playback Cycle

import { PlaybackEngine } from '@/lib/playback/engine';
import { ActionEngine } from '@/lib/action/engine';
import { AudioPlayer } from '@/lib/utils/audio-player';

const engine = new PlaybackEngine(
  scenes,           // Array of Stage scenes
  new ActionEngine(),
  new AudioPlayer(),
  {
    onModeChange: (mode) => console.log('Engine mode →', mode),
    onProgress: (snapshot) => console.log('Progress:', snapshot),
  },
);

engine.start();   // idle → playing
engine.pause();   // playing → paused
engine.resume();  // paused → playing

Proactive Q&A Flow

const engine = new PlaybackEngine(scenes, actionEngine, audioPlayer, {
  onProactiveShow: (trigger) => {
    // Render discussion card in UI
    console.log('Proactive question:', trigger.question);
  },
  onDiscussionConfirmed: (topic, prompt, agentId) => {
    console.log('Confirmed discussion on:', topic);
  },
});

// User clicks "Join" on the card
engine.confirmDiscussion();  // playing → live

Handling Chat Interruptions

// Student types during lecture playback
engine.handleUserInterrupt('Can you explain the transformer architecture?');
// playing → live (cursor saved automatically)

// Later, end discussion and return to lecture
engine.handleEndDiscussion();  // live → idle

Key Implementation Files

File	Responsibility
[lib/playback/engine.ts](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/engine.ts)	PlaybackEngine class, state machine diagram, all transition methods
[lib/playback/types.ts](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/types.ts)	EngineMode, TopicState, PlaybackEngineCallbacks definitions
[lib/playback/derived-state.ts](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/derived-state.ts)	Stage state consolidation for UI consumption
[tests/playback/action-navigation.test.ts](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/playback/action-navigation.test.ts)	Unit tests verifying navigation and transitions

Summary

• The classroom playback state machine centers on a four-mode PlaybackEngine with deterministic transitions between idle, playing, paused, and live.

• Live interaction integrates seamlessly through confirmDiscussion() and handleUserInterrupt(), both preserving lecture cursor position for clean resumption.

• Topic state (active/pending/closed) extends the engine mode to handle paused discussions without data loss.

• The callback-driven architecture in PlaybackEngineCallbacks maintains strict separation between state logic and UI components.

• All source references originate from lib/playback in the THU-MAIC/OpenMAIC repository, with the state machine diagram explicitly documented in engine.ts lines 7–23.

Frequently Asked Questions

How does the engine handle unexpected chat messages during playback?

The handleUserInterrupt() method captures unsolicited student input by immediately saving the current lecture cursor, aborting active playback timers and TTS, and transitioning playing or paused → live. The lecture position remains stored for restoration via handleEndDiscussion().

What happens if a user pauses during a live discussion?

Calling pause() while in live mode transitions to paused but leaves the topic in pending state. A subsequent resume() detects this pending status and returns to live mode rather than resuming lecture playback, preserving the discussion context.

Where is the state machine visually documented in the source code?

The complete state machine diagram appears as an ASCII comment block in [lib/playback/engine.ts](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/engine.ts) starting at line 7. This diagram illustrates all valid transitions between idle, playing, paused, and live modes, including the confirmDiscussion() and handleUserInterrupt() entry points into live discussion.

How does the engine recover from failed live discussions?

The handleDiscussionError() method provides a universal reset path from any state back to idle. This handles network failures, agent timeouts, or other live discussion errors by clearing all transient state and returning the engine to a known-good initial condition.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}