ActionEngine in OpenMAIC: How to Execute and Control Interactive Scenes

The ActionEngine in OpenMAIC is the core runtime class that schedules and executes sequences of interactive actions, supporting auto-play, manual stepping, pause/resume with state persistence, and safe navigation between speech actions.

The ActionEngine drives all interactive behaviors in OpenMAIC lessons. Defined in lib/action/engine.ts, it receives arrays of actions—such as speech synthesis, spotlight highlighting, and widget updates—and executes them in order while handling timing, silent flags, and recovery from interruptions.

What Is the ActionEngine?

The ActionEngine serves as the central execution runtime for OpenMAIC scenes. A scene bundles a list of actions, and the engine processes these actions sequentially according to their defined types. Each action can represent spoken dialogue, visual highlighting (spotlights), laser pointers, whiteboard drawings, or UI widget updates.

Key responsibilities include:

• Scheduling and Execution – The engine invokes the appropriate handler for each action type and waits for completion before continuing.
• Resume State Management – It tracks the current action index to support pausing and resuming across page reloads.
• Safety Checks – Navigation helpers ensure jumps only land on safe, renderable actions (typically speech actions).
• Silent Execution – Optional flags allow actions to run without audible side effects, useful for background syncing.

Core Files and Architecture

The ActionEngine implementation is split across several modules in the lib/action/ directory:

• lib/action/engine.ts – Contains the main ActionEngine class declaration starting at line 178. This file defines the constructor, run(), execute(), pause(), and seek() methods.
• lib/action/resume.ts – Exports helpers for persisting playback position, including createActionResumePosition, saveActionResumePosition, and readActionResumeState.
• lib/action/navigation.ts – Provides utilities such as getPreviousSafeSpeechActionIndex and getNextSafeSpeechActionIndex for building UI navigation controls that skip non-speech actions.

Test suites demonstrating usage patterns are available in tests/playback/action-navigation.test.ts and tests/playback/discussion-consumption.test.ts.

Instantiating the ActionEngine

Create an engine instance by passing a flat array of Action objects to the constructor. The engine prepares internal indices and resume state immediately upon instantiation.

import { ActionEngine } from '@/lib/action/engine';
import type { Action } from '@openmaic/dsl';

// Helper to create a speech action
const speech = (id: string, text: string): Action =>
  ({ id, type: 'speech', text }) as Action;

// Define scene actions
const sceneActions: Action[] = [
  speech('intro', 'Welcome to the lesson!'),
  { id: 'spot-1', type: 'spotlight', elementId: 'box' } as Action,
  speech('next', 'Let us dive deeper.'),
];

// Instantiate the engine
const engine = new ActionEngine(sceneActions);

Executing Actions

The ActionEngine supports both automatic playback and manual stepping.

Auto-Play Mode with run()

Call run() to execute the entire remaining sequence automatically. The method returns a Promise that resolves when the final action completes.

// Run all actions from the current position
await engine.run();

Manual Execution with execute()

Use execute() to run a specific single action. This is useful for custom UI controls or testing specific behaviors. Pass an options object with silent: true to suppress audio output.

// Execute the first action
await engine.execute(sceneActions[0]);

// Execute spotlight silently (no audio)
await engine.execute(sceneActions[1], { silent: true });

Managing Resume State and Persistence

To survive page reloads or navigation events, use the resume utilities from lib/action/resume.ts. These functions serialize the current playback position to a storage backend and restore it later.

import {
  createActionResumePosition,
  saveActionResumePosition,
  readActionResumeState,
  getValidActionResumePosition,
} from '@/lib/action/resume';

// Pause and save the current position
const pausePos = engine.getCurrentPosition();
await saveActionResumePosition(storage, 'user-key', 'scene-42', pausePos);

// Later, restore and resume
const state = await readActionResumeState(storage, 'user-key');
const resumePos = getValidActionResumePosition(state, 'scene-42', sceneActions);

if (resumePos) {
  engine.seek(resumePos.actionIndex);
  await engine.run(); // Continues from saved spot
}

Safe Navigation Between Actions

When building previous/next controls, you must skip actions that are not safe to render (such as invisible transitions or metadata markers). The navigation utilities compute valid indices that target only speech actions.

import {
  buildActionNavigationTargets,
  getPreviousSafeSpeechActionIndex,
  getNextSafeSpeechActionIndex,
} from '@/lib/action/navigation';

const targets = buildActionNavigationTargets(sceneActions);
// targets contains UI-ready data like timestamps and labels

const prevIdx = getPreviousSafeSpeechActionIndex(sceneActions, currentIdx);
const nextIdx = getNextSafeSpeechActionIndex(sceneActions, currentIdx);

These functions filter the action list to ensure navigation jumps only to indices containing valid speech content.

Summary

• The ActionEngine class is defined at line 178 of lib/action/engine.ts and serves as the primary runtime for OpenMAIC scenes.
• Instantiate the engine with new ActionEngine(actions), passing an array of Action objects.
• Use await engine.run() for automatic sequential playback through the end of the scene.
• Use await engine.execute(action, options) for manual single-step execution; pass { silent: true } to mute audio.
• Handle interruptions using resume helpers in lib/action/resume.ts, which persist state via saveActionResumePosition and restore it with readActionResumeState.
• Implement navigation controls using utilities from lib/action/navigation.ts to calculate safe previous and next indices that skip non-speech actions.

Frequently Asked Questions

Where is the ActionEngine class defined in the OpenMAIC repository?

The ActionEngine class is defined in lib/action/engine.ts, beginning at line 178. This file contains the constructor, execution methods, and pause/resume logic.

How does the ActionEngine handle interruptions and resume playback?

The engine tracks the current action index internally. When pausing, you retrieve the position with engine.getCurrentPosition(), then persist it using saveActionResumePosition. To resume, use readActionResumeState to fetch the saved data, validate it with getValidActionResumePosition, seek to the index with engine.seek(), and call run() to continue.

What is the difference between run() and execute()?

run() automatically iterates through the action list from the current position until completion, handling each action in sequence. execute(action, options) runs a single specific action immediately, allowing you to control timing manually and pass options such as silent: true to disable audio feedback for that specific step.

Can the ActionEngine skip non-speech actions during navigation?

Yes. The navigation utilities getPreviousSafeSpeechActionIndex and getNextSafeSpeechActionIndex in lib/action/navigation.ts filter the action array to return only indices containing speech actions. This ensures that previous/next buttons in the UI always jump to renderable content, skipping invisible transitions or metadata markers.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}