How Action-Level Playback Navigation Works in the OpenMAIC Classroom Player

Action-level playback navigation in the OpenMAIC classroom player enables teachers to jump between individual speech actions while blocking unsafe actions like video playback or widget state changes, using a deterministic safety filter implemented in lib/playback/action-navigation.ts.

The OpenMAIC classroom player provides granular playback controls that allow educators to navigate recorded lessons at the level of individual speech actions. Unlike standard video scrubbing, this action-level playback navigation system restricts jumps to "safe" actions that can be reconstructed without side effects, ensuring consistent lesson state. This architecture prevents navigation to actions that depend on external state or prior visual modifications that cannot be easily rewound.

What Is Action-Level Playback Navigation?

Action-level playback navigation allows teachers to move through a recorded lesson by jumping between specific speech actions rather than scrubbing through a timeline. The system analyzes the full action list of a lesson and determines which speech actions are safe to jump to based on whether all preceding actions can be safely reconstructed. Navigation is intentionally limited to "safe" speech actions—those appearing before any unsafe actions that might create irreversible side effects like video playback or widget state modifications.

Core Architecture and Components

The navigation system consists of three primary components that work together to provide safe, deterministic playback control.

The Action Navigation Module

The core logic lives in lib/playback/action-navigation.ts, which exports helper functions to analyze action sequences and determine safe jump targets. This module is responsible for identifying unsafe action types, building navigation targets, and calculating progress indicators. It provides the buildActionNavigationTargets, canReconstructPrefixForAction, and canJumpWithinReconstructablePrefix functions that form the safety layer.

The Playback Engine Integration

The lib/playback/engine.ts file consumes these navigation helpers to drive playback state. On line 47 of engine.ts, the engine imports functions like getPreviousSafeSpeechActionIndex, getNextSafeSpeechActionIndex, and getActionLineProgress. The engine queries these helpers to decide which actions can be jumped to when users click navigation controls, ensuring that all jumps respect the reconstructable prefix constraint.

Classroom Player UI Controls

The UI components render navigation buttons (previous/next) and progress indicators by invoking the engine's jump methods indirectly. These controls rely on the navigation helpers to determine button disabled states and progress bar calculations.

How Safe Navigation Targets Are Determined

The navigation system uses a whitelist approach to identify which actions can serve as valid jump targets.

Identifying Unsafe Action Types

The system defines a set of action types that block navigation because replaying them may depend on external state or cause side effects:

const UNSAFE_ACTION_TYPES = new Set<Action['type']>([
  'play_video',
  'discussion',
  'widget_highlight',
  'widget_setState',
  'widget_annotation',
  'widget_reveal',
]);

Any action whose type appears in UNSAFE_ACTION_TYPES is blocked for navigation. This prevents teachers from jumping to points in the lesson where a video has already played or a widget has been modified, as these states cannot be easily reconstructed.

Building Navigation Targets with buildActionNavigationTargets

The buildActionNavigationTargets function walks the action list, filters only speech actions, assigns line numbers, and marks whether each action can be jumped to:

export function buildActionNavigationTargets(actions: readonly Action[]): ActionNavigationTarget[] {
  let lineNumber = 0;
  return actions.flatMap((action, actionIndex) => {
    if (action.type !== 'speech') return [];
    lineNumber += 1;
    return [{
      actionIndex,
      actionId: action.id,
      actionType: action.type,
      lineNumber,
      canJump: canReconstructPrefixForAction(actions, actionIndex),
    }];
  });
}

This function returns an array of navigation targets, where canJump indicates whether it is safe to jump to that specific action.

Validating Reconstructable Prefixes with canReconstructPrefixForAction

The canReconstructPrefixForAction function (implemented in lines 48‑63 of lib/playback/action-navigation.ts) returns true only if all preceding actions are safe and the current action itself is a speech action. This ensures that the player can reconstruct the entire lesson state up to the target action without encountering any unsafe operations.

Implementing Navigation Controls

The navigation helpers enable forward and backward movement through the transcript while maintaining safety constraints.

Jumping Forward and Backward Between Safe Actions

Two primary helpers expose the next and previous safe speech action indices relative to the current cursor:

export function getPreviousSafeSpeechActionIndex(actions, currentActionIndex) { … }
export function getNextSafeSpeechActionIndex(actions, currentActionIndex) { … }

Both functions locate the nearest navigation target whose canJump flag is true and whose index lies before or after the current cursor. The engine uses these to implement the previous and next button functionality.

Calculating Transcript Progress

The getActionLineProgress function computes the transcript line to display for the current playback position:

export function getActionLineProgress(actions, currentActionIndex) {
  const targets = buildActionNavigationTargets(actions);
  const cursor = Math.max(0, currentActionIndex ?? 0);
  const exactOrPrevious = [...targets].reverse().find(t => t.actionIndex <= cursor);
  const currentLine = exactOrPrevious?.lineNumber ?? targets[0].lineNumber;
  return { currentLine, totalLines: targets.length };
}

The UI displays currentLine / totalLines, giving teachers a clear sense of their position in the lesson transcript. The function searches backwards through targets to find the closest speech action at or before the current cursor position.

Code Implementation Examples

Example 1: Getting the Next Safe Speech Action

import { getNextSafeSpeechActionIndex } from '@/lib/playback/action-navigation';

// `actions` is the full lesson action array, `cursor` is the current action index.
const nextIdx = getNextSafeSpeechActionIndex(actions, cursor);
if (nextIdx !== null) {
  // Jump to the next safe speech action.
  playbackEngine.jumpTo(nextIdx);
}

Example 2: Rendering the Progress Bar

import { getActionLineProgress } from '@/lib/playback/action-navigation';

function PlaybackProgress({ actions, currentIdx }: { actions: Action[]; currentIdx: number }) {
  const { currentLine, totalLines } = getActionLineProgress(actions, currentIdx);
  return (
    <div className="progress">
      {currentLine} / {totalLines}
    </div>
  );
}

Example 3: Disabling Navigation Buttons

const prevIdx = getPreviousSafeSpeechActionIndex(actions, cursor);
const nextIdx = getNextSafeSpeechActionIndex(actions, cursor);

return (
  <>
    <button disabled={prevIdx === null} onClick={() => prevIdx && engine.jumpTo(prevIdx)}>Prev</button>
    <button disabled={nextIdx === null} onClick={() => nextIdx && engine.jumpTo(nextIdx)}>Next</button>
  </>
);

Key Files and Dependencies

• lib/playback/action-navigation.ts: Core logic for safe speech-action navigation, building targets, and progress calculations.
• lib/playback/engine.ts: Playback engine that consumes navigation helpers on line 47 to implement player controls.
• tests/playback/action-navigation.test.ts: Comprehensive test suite verifying navigation behavior across various action sequences.
• UI component files: Classroom player control components that call the engine methods (located in the components directory).

Summary

• Action-level playback navigation restricts jumping to specific speech actions that can be safely reconstructed without side effects.
• The system defines unsafe action types (video playback, widget modifications, discussions) that block navigation to prevent state inconsistencies.
• The buildActionNavigationTargets function in lib/playback/action-navigation.ts constructs valid jump targets by filtering speech actions and validating reconstructable prefixes.
• Navigation helpers getPreviousSafeSpeechActionIndex and getNextSafeSpeechActionIndex locate the nearest safe jump points relative to the current cursor.
• The getActionLineProgress function provides transcript line tracking by searching backwards through navigation targets to find the current speech position.

Frequently Asked Questions

What makes an action "unsafe" for navigation in the OpenMAIC player?

An action is considered unsafe if its type appears in the UNSAFE_ACTION_TYPES set, which includes play_video, discussion, widget_highlight, widget_setState, widget_annotation, and widget_reveal. These actions are blocked because they may depend on external state or create side effects that cannot be reconstructed when jumping backwards or forwards in the lesson timeline.

How does the player determine the current line number in the transcript?

The player uses the getActionLineProgress function to calculate the current line by first building all navigation targets, then searching backwards through them to find the closest speech action at or before the current cursor position. This returns both the current line number and the total count of speech actions in the lesson.

Can teachers jump to any speech action regardless of its position in the lesson?

No, teachers can only jump to speech actions that have the canJump flag set to true by the buildActionNavigationTargets function. This flag is only set if canReconstructPrefixForAction confirms that all preceding actions are safe, meaning the target appears before any unsafe actions like video playback or widget state changes in the sequence.

What happens if there is no safe previous or next action available?

The navigation helper functions getPreviousSafeSpeechActionIndex and getNextSafeSpeechActionIndex return null when no safe target exists in the requested direction. The UI should check for null and disable the corresponding navigation button, preventing the teacher from jumping to an invalid or unsafe lesson state.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}