# ActionEngine in OpenMAIC: How to Execute and Control Interactive Scenes

> Explore OpenMAIC's ActionEngine, the core runtime for executing interactive scenes. Learn to control auto-play, manual steps, and state persistence for seamless scene navigation.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: how-to-guide
- Published: 2026-08-30

---

**The ActionEngine in OpenMAIC is the core runtime class that schedules and executes sequences of interactive actions, supporting auto-play, manual stepping, pause/resume with state persistence, and safe navigation between speech actions.**

The ActionEngine drives all interactive behaviors in OpenMAIC lessons. Defined in [`lib/action/engine.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/engine.ts), it receives arrays of actions—such as speech synthesis, spotlight highlighting, and widget updates—and executes them in order while handling timing, silent flags, and recovery from interruptions.

## What Is the ActionEngine?

The ActionEngine serves as the central execution runtime for OpenMAIC scenes. A **scene** bundles a list of actions, and the engine processes these actions sequentially according to their defined types. Each action can represent spoken dialogue, visual highlighting (spotlights), laser pointers, whiteboard drawings, or UI widget updates.

Key responsibilities include:

- **Scheduling and Execution** – The engine invokes the appropriate handler for each action type and waits for completion before continuing.
- **Resume State Management** – It tracks the current action index to support pausing and resuming across page reloads.
- **Safety Checks** – Navigation helpers ensure jumps only land on safe, renderable actions (typically speech actions).
- **Silent Execution** – Optional flags allow actions to run without audible side effects, useful for background syncing.

## Core Files and Architecture

The ActionEngine implementation is split across several modules in the `lib/action/` directory:

- **[`lib/action/engine.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/engine.ts)** – Contains the main `ActionEngine` class declaration starting at line 178. This file defines the constructor, `run()`, `execute()`, `pause()`, and `seek()` methods.
- **[`lib/action/resume.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/resume.ts)** – Exports helpers for persisting playback position, including `createActionResumePosition`, `saveActionResumePosition`, and `readActionResumeState`.
- **[`lib/action/navigation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/navigation.ts)** – Provides utilities such as `getPreviousSafeSpeechActionIndex` and `getNextSafeSpeechActionIndex` for building UI navigation controls that skip non-speech actions.

Test suites demonstrating usage patterns are available in [`tests/playback/action-navigation.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/playback/action-navigation.test.ts) and [`tests/playback/discussion-consumption.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/playback/discussion-consumption.test.ts).

## Instantiating the ActionEngine

Create an engine instance by passing a flat array of `Action` objects to the constructor. The engine prepares internal indices and resume state immediately upon instantiation.

```typescript
import { ActionEngine } from '@/lib/action/engine';
import type { Action } from '@openmaic/dsl';

// Helper to create a speech action
const speech = (id: string, text: string): Action =>
  ({ id, type: 'speech', text }) as Action;

// Define scene actions
const sceneActions: Action[] = [
  speech('intro', 'Welcome to the lesson!'),
  { id: 'spot-1', type: 'spotlight', elementId: 'box' } as Action,
  speech('next', 'Let us dive deeper.'),
];

// Instantiate the engine
const engine = new ActionEngine(sceneActions);

```

## Executing Actions

The ActionEngine supports both automatic playback and manual stepping.

### Auto-Play Mode with run()

Call `run()` to execute the entire remaining sequence automatically. The method returns a Promise that resolves when the final action completes.

```typescript
// Run all actions from the current position
await engine.run();

```

### Manual Execution with execute()

Use `execute()` to run a specific single action. This is useful for custom UI controls or testing specific behaviors. Pass an options object with `silent: true` to suppress audio output.

```typescript
// Execute the first action
await engine.execute(sceneActions[0]);

// Execute spotlight silently (no audio)
await engine.execute(sceneActions[1], { silent: true });

```

## Managing Resume State and Persistence

To survive page reloads or navigation events, use the resume utilities from [`lib/action/resume.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/resume.ts). These functions serialize the current playback position to a storage backend and restore it later.

```typescript
import {
  createActionResumePosition,
  saveActionResumePosition,
  readActionResumeState,
  getValidActionResumePosition,
} from '@/lib/action/resume';

// Pause and save the current position
const pausePos = engine.getCurrentPosition();
await saveActionResumePosition(storage, 'user-key', 'scene-42', pausePos);

// Later, restore and resume
const state = await readActionResumeState(storage, 'user-key');
const resumePos = getValidActionResumePosition(state, 'scene-42', sceneActions);

if (resumePos) {
  engine.seek(resumePos.actionIndex);
  await engine.run(); // Continues from saved spot
}

```

## Safe Navigation Between Actions

When building previous/next controls, you must skip actions that are not safe to render (such as invisible transitions or metadata markers). The navigation utilities compute valid indices that target only speech actions.

```typescript
import {
  buildActionNavigationTargets,
  getPreviousSafeSpeechActionIndex,
  getNextSafeSpeechActionIndex,
} from '@/lib/action/navigation';

const targets = buildActionNavigationTargets(sceneActions);
// targets contains UI-ready data like timestamps and labels

const prevIdx = getPreviousSafeSpeechActionIndex(sceneActions, currentIdx);
const nextIdx = getNextSafeSpeechActionIndex(sceneActions, currentIdx);

```

These functions filter the action list to ensure navigation jumps only to indices containing valid speech content.

## Summary

- The `ActionEngine` class is defined at line 178 of [`lib/action/engine.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/engine.ts) and serves as the primary runtime for OpenMAIC scenes.
- Instantiate the engine with `new ActionEngine(actions)`, passing an array of Action objects.
- Use `await engine.run()` for automatic sequential playback through the end of the scene.
- Use `await engine.execute(action, options)` for manual single-step execution; pass `{ silent: true }` to mute audio.
- Handle interruptions using resume helpers in [`lib/action/resume.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/resume.ts), which persist state via `saveActionResumePosition` and restore it with `readActionResumeState`.
- Implement navigation controls using utilities from [`lib/action/navigation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/navigation.ts) to calculate safe previous and next indices that skip non-speech actions.

## Frequently Asked Questions

### Where is the ActionEngine class defined in the OpenMAIC repository?

The `ActionEngine` class is defined in [`lib/action/engine.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/engine.ts), beginning at line 178. This file contains the constructor, execution methods, and pause/resume logic.

### How does the ActionEngine handle interruptions and resume playback?

The engine tracks the current action index internally. When pausing, you retrieve the position with `engine.getCurrentPosition()`, then persist it using `saveActionResumePosition`. To resume, use `readActionResumeState` to fetch the saved data, validate it with `getValidActionResumePosition`, seek to the index with `engine.seek()`, and call `run()` to continue.

### What is the difference between run() and execute()?

`run()` automatically iterates through the action list from the current position until completion, handling each action in sequence. `execute(action, options)` runs a single specific action immediately, allowing you to control timing manually and pass options such as `silent: true` to disable audio feedback for that specific step.

### Can the ActionEngine skip non-speech actions during navigation?

Yes. The navigation utilities `getPreviousSafeSpeechActionIndex` and `getNextSafeSpeechActionIndex` in [`lib/action/navigation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/navigation.ts) filter the action array to return only indices containing speech actions. This ensures that previous/next buttons in the UI always jump to renderable content, skipping invisible transitions or metadata markers.