# How the OpenMAIC Playback Engine Manages Classroom States: A Deep Dive into the State Machine

> Discover how the OpenMAIC playback engine manages classroom states using a finite state machine. Explore idle, playing, paused, and live modes for seamless lecture playback and interactive discussions.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: deep-dive
- Published: 2026-08-31

---

**The OpenMAIC playback engine manages classroom states through a finite state machine with four mutually exclusive modes—`idle`, `playing`, `paused`, and `live`—that transitions between recorded lecture playback and live interactive discussions while preserving cursor position and generation state.**

The `PlaybackEngine` in the [THU-MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC) repository serves as the central nervous system for AI-driven classrooms, orchestrating the seamless transition between pre-recorded lecture segments and real-time student interactions. By maintaining a single source of truth through `sceneIndex`, `actionIndex`, and `mode` properties, the engine ensures that user interruptions, discussion prompts, and playback controls behave predictably across the application lifecycle.

## Core State Machine Architecture

The engine defines four discrete states that govern all classroom activity. According to [`lib/playback/engine.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/engine.ts) lines 9-23, these modes are mutually exclusive and enforce strict transition rules:

- **`idle`**: The initial state before playback begins or after a discussion ends
- **`playing`**: Active lecture playback processing actions sequentially  
- **`paused`**: Temporary halt of playback timers and audio
- **`live`**: Interactive discussion mode triggered by user interrupts or proactive discussion actions

The state diagram permits specific transitions such as `idle → playing`, `playing → paused`, and `paused → live` following a user interrupt, preventing invalid state combinations that could corrupt the lecture cursor.

## Starting and Controlling Playback

The public API provides distinct entry points depending on whether the lecture is beginning fresh or resuming from an interrupted state.

### Fresh Lecture Initialization

The `start()` method (lines 62-80 in [`engine.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/engine.ts)) performs a complete reset of the playback context:

```typescript
playback.start(); // Transitions: idle → playing

```

This operation resets the cursor to the beginning of the lecture, increments the generation marker to invalidate stale timers, and begins processing the first action in the sequence.

### Resuming After Interruptions

When a live discussion concludes, `continuePlayback()` (lines 63-71) re-enters the **playing** state without resetting the cursor, allowing the lecture to resume exactly where the interruption occurred:

```typescript
playback.continuePlayback(); // live → playing (cursor preserved)

```

This distinction is critical for maintaining pedagogical continuity—unlike `start()`, this method retains the `sceneIndex` and `actionIndex` from the pre-interruption state.

## Handling User Interruptions

When learners initiate chat during playback, `handleUserInterrupt()` (lines 40-68) executes a coordinated state transition:

1. Saves the current lecture cursor position
2. Cancels pending triggers and timers via generation invalidation
3. Switches mode to **live**
4. Stops audio playback and browser-native TTS
5. Notifies the UI through callbacks

```typescript
// During active playback, user sends a message
playback.handleUserInterrupt('Can you explain this part again?');
// Engine immediately: saves cursor → cancels timers → live mode

```

This ensures the lecture context is preserved while the system transitions to interactive mode.

## Discussion Lifecycle Management

The engine implements a sophisticated three-phase discussion protocol with specific timing and state requirements.

### Proactive Discussion Triggers

When the engine encounters a `discussion` action, it initiates a 3-second delay (`DISCUSSION_TRIGGER_DELAY_MS`) before emitting the `onProactiveShow` callback (lines 98-115). This grace period allows the UI to prepare discussion cards without interrupting the flow of the lecture abruptly.

### Discussion Confirmation and Skipping

Once the proactive discussion appears, the UI layer invokes one of two methods:

- **`confirmDiscussion()`**: Marks the discussion as consumed, saves the lecture state to the cursor, transitions to **live** mode, and fires `onDiscussionConfirmed` callbacks with question, payload, and answer parameters
- **`skipDiscussion()`**: Records consumption and proceeds to the next action without entering live mode

Both methods prevent re-triggering of the same discussion point through internal consumption tracking.

### Ending Discussions

The `handleEndDiscussion()` method (lines 98-114) manages the cleanup sequence:

```typescript
playback.handleEndDiscussion();
// Restores saved lecture cursor → clears whiteboard → onDiscussionEnd → idle

```

This restores the saved lecture position, resets the whiteboard state, and transitions the engine back to **idle**, preparing it for either `continuePlayback()` or full termination.

## Error Handling and Safety Guards

### Discussion Error Recovery

If a live discussion fails due to network errors or AI service unavailability, `handleDiscussionError()` (lines 116-136) aborts the live mode, restores the lecture cursor from the saved checkpoint, and returns the engine to **idle**, preventing the classroom from becoming stuck in a broken live state.

### Generation Guards Against Race Conditions

All timer-based callbacks validate the current generation ID through `isCurrentGeneration()` (lines 93-100). When users jump to new actions or stop playback, the generation increments, causing stale asynchronous operations to self-terminate rather than executing against invalid state.

### Exhaustion Detection

The `isExhausted()` method (lines 70-89) iterates through remaining actions and treats already-consumed discussions as completed, enabling the engine to accurately determine when the lecture has fully played rather than waiting indefinitely for skipped interactions.

## Implementation Example

The following pattern demonstrates typical integration within a classroom workbench component:

```typescript
import { PlaybackEngine } from '@/lib/playback/engine';
import { ActionEngine } from '@/lib/action/engine';
import { AudioPlayer } from '@/lib/utils/audio-player';

// Initialize with lecture scenes and callbacks
const playback = new PlaybackEngine(scenes, actionEngine, audioPlayer, {
  onModeChange: (mode) => console.log('Mode →', mode),
  onProgress: (snap) => console.log('Progress', snap),
  onProactiveShow: (trigger) => showDiscussionCard(trigger),
  onDiscussionConfirmed: (q, p, a) => startLiveChat(q, p, a),
});

// Standard lecture flow
playback.start();                    // Begin lecture
playback.pause();                    // User pauses
playback.resume();                   // Resume playback

// Interactive interruption flow
playback.handleUserInterrupt('Question about theorem 3');
// ... discussion occurs ...
playback.handleEndDiscussion();      // Return to idle
playback.continuePlayback();         // Resume lecture

```

## Key Source Files

The classroom state management system spans several specialized modules:

- **[`lib/playback/engine.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/engine.ts)**: Central state machine implementing `PlaybackEngine` class with mode transitions and public API
- **[`lib/playback/types.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/types.ts)**: TypeScript definitions for `EngineMode`, `PlaybackSnapshot`, and callback interfaces
- **[`lib/playback/cursor.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/cursor.ts)**: Cursor management utilities for resolving current actions and advancing scene boundaries
- **[`lib/playback/derived-state.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/derived-state.ts)**: Computed state logic including pending discussion detection
- **[`lib/playback/auto-resume.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/auto-resume.ts)**: Automatic resumption logic post-interruption
- **[`lib/playback/action-navigation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/action-navigation.ts)**: Action jumping utilities with reconstructable prefix validation

## Summary

- The OpenMAIC playback engine maintains classroom states through a strict four-mode finite state machine (`idle`, `playing`, `paused`, `live`) defined in [`lib/playback/engine.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/engine.ts)
- **Cursor preservation** during interruptions ensures lectures resume at the exact point of discussion, managed through `handleUserInterrupt()` and `continuePlayback()`
- **Generation guards** prevent race conditions by invalidating stale timers when users navigate or stop playback abruptly
- The **discussion lifecycle** includes 3-second proactive delays, explicit confirmation/skip handling, and clean return-to-playback protocols
- **Error boundaries** in `handleDiscussionError()` ensure the engine fails safely back to `idle` rather than corrupting the lecture state

## Frequently Asked Questions

### How does the OpenMAIC playback engine handle simultaneous user interrupts?

The engine processes interrupts sequentially through `handleUserInterrupt()`, which immediately cancels pending timers via generation invalidation and switches to **live** mode. If another interrupt arrives while already in **live** mode, the current discussion context takes precedence, and the new interrupt is queued or handled according to the application layer's chat implementation.

### What happens to audio playback when transitioning from playing to live mode?

When `handleUserInterrupt()` executes, it explicitly stops the browser-native TTS (Text-to-Speech) and halts audio playback before entering **live** mode. The `pause()` method similarly stops timers and audio when invoked during **playing** state, ensuring no background lecture audio interferes with live discussion.

### Can the OpenMAIC playback engine resume from the middle of a specific scene?

Yes, through `continuePlayback()`, the engine resumes from the exact cursor position saved prior to interruption. The cursor—managed in [`lib/playback/cursor.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/cursor.ts)—tracks both `sceneIndex` and `actionIndex`, allowing resumption mid-scene without requiring the user to restart the entire lecture segment.

### How does the engine prevent discussion actions from triggering twice?

The engine marks discussions as "consumed" when either `confirmDiscussion()` or `skipDiscussion()` is called. The `isExhausted()` method treats consumed discussions as completed actions, and subsequent playback iterations skip these actions, preventing duplicate triggers even if the user navigates backward and forward through the lecture timeline.