How the OpenMAIC Playback Engine Manages Classroom States: A Deep Dive into the State Machine

The OpenMAIC playback engine manages classroom states through a finite state machine with four mutually exclusive modes—idle, playing, paused, and live—that transitions between recorded lecture playback and live interactive discussions while preserving cursor position and generation state.

The PlaybackEngine in the THU-MAIC/OpenMAIC repository serves as the central nervous system for AI-driven classrooms, orchestrating the seamless transition between pre-recorded lecture segments and real-time student interactions. By maintaining a single source of truth through sceneIndex, actionIndex, and mode properties, the engine ensures that user interruptions, discussion prompts, and playback controls behave predictably across the application lifecycle.

Core State Machine Architecture

The engine defines four discrete states that govern all classroom activity. According to lib/playback/engine.ts lines 9-23, these modes are mutually exclusive and enforce strict transition rules:

• idle: The initial state before playback begins or after a discussion ends
• playing: Active lecture playback processing actions sequentially
• paused: Temporary halt of playback timers and audio
• live: Interactive discussion mode triggered by user interrupts or proactive discussion actions

The state diagram permits specific transitions such as idle → playing, playing → paused, and paused → live following a user interrupt, preventing invalid state combinations that could corrupt the lecture cursor.

Starting and Controlling Playback

The public API provides distinct entry points depending on whether the lecture is beginning fresh or resuming from an interrupted state.

Fresh Lecture Initialization

The start() method (lines 62-80 in engine.ts) performs a complete reset of the playback context:

playback.start(); // Transitions: idle → playing

This operation resets the cursor to the beginning of the lecture, increments the generation marker to invalidate stale timers, and begins processing the first action in the sequence.

Resuming After Interruptions

When a live discussion concludes, continuePlayback() (lines 63-71) re-enters the playing state without resetting the cursor, allowing the lecture to resume exactly where the interruption occurred:

playback.continuePlayback(); // live → playing (cursor preserved)

This distinction is critical for maintaining pedagogical continuity—unlike start(), this method retains the sceneIndex and actionIndex from the pre-interruption state.

Handling User Interruptions

When learners initiate chat during playback, handleUserInterrupt() (lines 40-68) executes a coordinated state transition:

1. Saves the current lecture cursor position
2. Cancels pending triggers and timers via generation invalidation
3. Switches mode to live
4. Stops audio playback and browser-native TTS
5. Notifies the UI through callbacks

// During active playback, user sends a message
playback.handleUserInterrupt('Can you explain this part again?');
// Engine immediately: saves cursor → cancels timers → live mode

This ensures the lecture context is preserved while the system transitions to interactive mode.

Discussion Lifecycle Management

The engine implements a sophisticated three-phase discussion protocol with specific timing and state requirements.

Proactive Discussion Triggers

When the engine encounters a discussion action, it initiates a 3-second delay (DISCUSSION_TRIGGER_DELAY_MS) before emitting the onProactiveShow callback (lines 98-115). This grace period allows the UI to prepare discussion cards without interrupting the flow of the lecture abruptly.

Discussion Confirmation and Skipping

Once the proactive discussion appears, the UI layer invokes one of two methods:

• confirmDiscussion(): Marks the discussion as consumed, saves the lecture state to the cursor, transitions to live mode, and fires onDiscussionConfirmed callbacks with question, payload, and answer parameters
• skipDiscussion(): Records consumption and proceeds to the next action without entering live mode

Both methods prevent re-triggering of the same discussion point through internal consumption tracking.

Ending Discussions

The handleEndDiscussion() method (lines 98-114) manages the cleanup sequence:

playback.handleEndDiscussion();
// Restores saved lecture cursor → clears whiteboard → onDiscussionEnd → idle

This restores the saved lecture position, resets the whiteboard state, and transitions the engine back to idle, preparing it for either continuePlayback() or full termination.

Error Handling and Safety Guards

Discussion Error Recovery

If a live discussion fails due to network errors or AI service unavailability, handleDiscussionError() (lines 116-136) aborts the live mode, restores the lecture cursor from the saved checkpoint, and returns the engine to idle, preventing the classroom from becoming stuck in a broken live state.

Generation Guards Against Race Conditions

All timer-based callbacks validate the current generation ID through isCurrentGeneration() (lines 93-100). When users jump to new actions or stop playback, the generation increments, causing stale asynchronous operations to self-terminate rather than executing against invalid state.

Exhaustion Detection

The isExhausted() method (lines 70-89) iterates through remaining actions and treats already-consumed discussions as completed, enabling the engine to accurately determine when the lecture has fully played rather than waiting indefinitely for skipped interactions.

Implementation Example

The following pattern demonstrates typical integration within a classroom workbench component:

import { PlaybackEngine } from '@/lib/playback/engine';
import { ActionEngine } from '@/lib/action/engine';
import { AudioPlayer } from '@/lib/utils/audio-player';

// Initialize with lecture scenes and callbacks
const playback = new PlaybackEngine(scenes, actionEngine, audioPlayer, {
  onModeChange: (mode) => console.log('Mode →', mode),
  onProgress: (snap) => console.log('Progress', snap),
  onProactiveShow: (trigger) => showDiscussionCard(trigger),
  onDiscussionConfirmed: (q, p, a) => startLiveChat(q, p, a),
});

// Standard lecture flow
playback.start();                    // Begin lecture
playback.pause();                    // User pauses
playback.resume();                   // Resume playback

// Interactive interruption flow
playback.handleUserInterrupt('Question about theorem 3');
// ... discussion occurs ...
playback.handleEndDiscussion();      // Return to idle
playback.continuePlayback();         // Resume lecture

Key Source Files

The classroom state management system spans several specialized modules:

• lib/playback/engine.ts: Central state machine implementing PlaybackEngine class with mode transitions and public API
• lib/playback/types.ts: TypeScript definitions for EngineMode, PlaybackSnapshot, and callback interfaces
• lib/playback/cursor.ts: Cursor management utilities for resolving current actions and advancing scene boundaries
• lib/playback/derived-state.ts: Computed state logic including pending discussion detection
• lib/playback/auto-resume.ts: Automatic resumption logic post-interruption
• lib/playback/action-navigation.ts: Action jumping utilities with reconstructable prefix validation

Summary

• The OpenMAIC playback engine maintains classroom states through a strict four-mode finite state machine (idle, playing, paused, live) defined in lib/playback/engine.ts
• Cursor preservation during interruptions ensures lectures resume at the exact point of discussion, managed through handleUserInterrupt() and continuePlayback()
• Generation guards prevent race conditions by invalidating stale timers when users navigate or stop playback abruptly
• The discussion lifecycle includes 3-second proactive delays, explicit confirmation/skip handling, and clean return-to-playback protocols
• Error boundaries in handleDiscussionError() ensure the engine fails safely back to idle rather than corrupting the lecture state

Frequently Asked Questions

How does the OpenMAIC playback engine handle simultaneous user interrupts?

The engine processes interrupts sequentially through handleUserInterrupt(), which immediately cancels pending timers via generation invalidation and switches to live mode. If another interrupt arrives while already in live mode, the current discussion context takes precedence, and the new interrupt is queued or handled according to the application layer's chat implementation.

What happens to audio playback when transitioning from playing to live mode?

When handleUserInterrupt() executes, it explicitly stops the browser-native TTS (Text-to-Speech) and halts audio playback before entering live mode. The pause() method similarly stops timers and audio when invoked during playing state, ensuring no background lecture audio interferes with live discussion.

Can the OpenMAIC playback engine resume from the middle of a specific scene?

Yes, through continuePlayback(), the engine resumes from the exact cursor position saved prior to interruption. The cursor—managed in lib/playback/cursor.ts—tracks both sceneIndex and actionIndex, allowing resumption mid-scene without requiring the user to restart the entire lecture segment.

How does the engine prevent discussion actions from triggering twice?

The engine marks discussions as "consumed" when either confirmDiscussion() or skipDiscussion() is called. The isExhausted() method treats consumed discussions as completed actions, and subsequent playback iterations skip these actions, preventing duplicate triggers even if the user navigates backward and forward through the lecture timeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}