OpenMAIC Action Types: Complete List of the Action Engine's Supported Verbs

OpenMAIC's action engine supports 20 distinct action types defined in the ACTION_TYPES constant, categorized into Fire-and-Forget visual effects, Synchronous blocking operations, and Widget Interactions, all strictly validated at runtime against the authoritative list in packages/@openmaic/dsl/src/action.ts.

OpenMAIC is an open-source AI-driven lesson generation platform that orchestrates interactive educational content through a unified action engine. The engine relies on a strictly defined set of action types to control everything from visual spotlights to complex widget state mutations. Understanding these supported action types is essential for developers building custom lesson content or extending the platform's capabilities.

The 20 Supported Action Types

The engine groups actions into three logical categories based on their execution behavior. All types are enumerated in the ACTION_TYPES tuple located in packages/@openmaic/dsl/src/action.ts (lines 90-113).

Fire-and-Forget Effects

These actions trigger immediate visual feedback without blocking subsequent operations:

• spotlight – Highlights a specific element while dimming the rest of the interface
• laser – Displays a laser pointer animation for emphasizing content

Synchronous Operations

These actions must complete before the lesson proceeds to the next step:

• speech – Text-to-speech audio narration
• play_video – Video playback control
• wb_open – Open the whiteboard/canvas
• wb_draw_text – Render text on the whiteboard
• wb_draw_shape – Draw geometric shapes
• wb_draw_chart – Render data visualizations
• wb_draw_latex – Display mathematical formulas
• wb_draw_table – Render tabular data
• wb_draw_line – Draw connector lines
• wb_draw_code – Display code blocks
• wb_edit_code – Modify existing code on the whiteboard
• wb_clear – Clear whiteboard content
• wb_delete – Remove specific whiteboard elements
• wb_close – Close the whiteboard
• discussion – Trigger discussion prompts or chat interactions

Widget Interactions

These operations manipulate embedded iframe widgets and simulations:

• widget_highlight – Highlight specific widget areas
• widget_setState – Update widget internal state (e.g., simulation parameters)
• widget_annotation – Add annotations to widget content
• widget_reveal – Show hidden widget components

Source Code Structure

The authoritative definition resides in packages/@openmaic/dsl/src/action.ts. The ACTION_TYPES constant uses TypeScript's satisfies operator to ensure compile-time exhaustiveness:

export const ACTION_TYPES = [
  'spotlight',
  'laser',
  'play_video',
  'speech',
  'wb_open',
  'wb_draw_text',
  'wb_draw_shape',
  'wb_draw_chart',
  'wb_draw_latex',
  'wb_draw_table',
  'wb_draw_line',
  'wb_draw_code',
  'wb_edit_code',
  'wb_clear',
  'wb_delete',
  'wb_close',
  'discussion',
  'widget_highlight',
  'widget_setState',
  'widget_annotation',
  'widget_reveal',
] as const satisfies readonly ActionType[];

This tuple guarantees that any action consumed by the engine must belong to this exhaustive set, enabling type-safe lesson definitions.

Runtime Validation Mechanism

The engine validates incoming actions at runtime using the isActionType type guard function defined in the same file. When processing a lesson, the system checks each action's type property against the ACTION_TYPES array.

According to the source code in packages/@openmaic/dsl/src/action.ts, this validation ensures forward compatibility by rejecting unknown verbs before they reach the execution pipeline. The validation is leveraged in lib/edit/content-validation.ts for editor-side checks and in the runtime engine for execution safety.

Practical Implementation Examples

The following examples demonstrate how to construct valid action objects for different categories using the @openmaic/dsl package.

Fire-and-Forget Spotlight Action

import { Action } from '@openmaic/dsl';

const spotlight: Action = {
  id: 'a1',
  type: 'spotlight',
  elementId: 'slide_3',
  dimOpacity: 0.4,
};

This action immediately spotlights slide_3 without pausing the lesson flow.

Synchronous Speech Action

import { Action } from '@openmaic/dsl';

const speech: Action = {
  id: 'a2',
  type: 'speech',
  text: 'Welcome to the lesson!',
  voice: 'en-US-Wavenet-D',
  speed: 1.0,
};

The engine waits for speech synthesis completion before proceeding to the next scene.

Widget State Mutation

import { Action } from '@openmaic/dsl';

const widgetSetState: Action = {
  id: 'a3',
  type: 'widget_setState',
  state: { temperature: 25, pressure: 101.3 },
  content: 'Adjusted simulation parameters.',
};

This updates an embedded simulation widget with new parameters while logging the change.

Key Files in the Action System

Understanding the action engine requires familiarity with these specific source files:

• packages/@openmaic/dsl/src/action.ts – Central type definitions, Action union type, ACTION_TYPES constant, and the isActionType validation helper
• packages/@openmaic/dsl/src/storage.ts – Defines AssetRef types used by speech and video actions for media references
• lib/server/agent-runtime/generation-tools.ts – AI-driven lesson creation logic that generates valid action sequences programmatically
• lib/edit/content-validation.ts – Editor-side validation that checks action objects against ACTION_TYPES before saving lessons

Summary

• OpenMAIC supports exactly 20 action types divided into three execution categories: Fire-and-Forget, Synchronous, and Widget Interactions.
• The canonical list is defined in packages/@openmaic/dsl/src/action.ts as the ACTION_TYPES tuple, providing compile-time type safety.
• Runtime validation via isActionType prevents unknown action types from entering the execution pipeline.
• Actions are plain JavaScript objects with a type property conforming to the Action union type, enabling seamless embedding in Stage and Scene hierarchies.
• Both online streaming and offline playback pipelines consume these action definitions without additional runtime dependencies.

Frequently Asked Questions

How does OpenMAIC validate action types during lesson execution?

The engine uses the isActionType type guard function to check incoming actions against the ACTION_TYPES array at runtime. This validation occurs in the content processing pipeline to ensure only defined verbs execute, preventing compatibility issues when lessons are played back in different runtime versions.

Can I add custom action types to the OpenMAIC engine?

Custom actions require modification of the source code in packages/@openmaic/dsl/src/action.ts. You must extend the ACTION_TYPES tuple and update the corresponding Action union type to include your new verb. The validation logic in lib/edit/content-validation.ts will automatically recognize new types once added to the central registry.

What is the difference between fire-and-forget and synchronous actions?

Fire-and-forget actions like spotlight and laser execute instantly without blocking the lesson timeline, allowing multiple effects to run simultaneously. Synchronous actions like speech and play_video halt progression until completion, ensuring temporal dependencies (such as narration finishing before text appears) are maintained correctly.

Which action types support whiteboard manipulation?

Whiteboard operations utilize twelve specific types: wb_open, wb_close, wb_clear, wb_delete, wb_draw_text, wb_draw_shape, wb_draw_chart, wb_draw_latex, wb_draw_table, wb_draw_line, wb_draw_code, and wb_edit_code. These provide comprehensive canvas control for interactive drawing and code demonstration scenarios.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}