# OpenMAIC Classroom Interaction Action Types: Complete Guide to lib/action

> Explore OpenMAIC lib/action to discover 22 classroom interaction action types. Master fire-and-forget effects, blocking operations, and widget interactions for your projects.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: api-reference
- Published: 2026-09-02

---

**The OpenMAIC platform supports 22 distinct classroom interaction action types through `lib/action`, organized into three execution categories: fire-and-forget visual effects, synchronous blocking operations, and widget interactions.**

The OpenMAIC classroom platform defines a unified **Action** contract that drives every classroom interaction event—from spotlighting elements to playing speech narration to updating interactive widgets. All concrete action shapes are declared in the DSL package `@openmaic/dsl` and re-exported through [`lib/types/action.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/types/action.ts), ensuring every subsystem (runtime engine, renderer, importer, and UI) shares a single source of truth.

## Action Type Categories in lib/action

The lib/action layer classifies classroom interaction action types by execution behavior. Understanding these categories is essential for lesson authors and platform developers working with the OpenMAIC source code.

### Fire-and-Forget Actions (Non-Blocking Visual Effects)

These **Action** types execute independently without blocking subsequent actions. They terminate immediately after dispatch and are ideal for subtle visual cues.

| Action Type | Interface | Source Location |
|-------------|-----------|---------------|
| `spotlight` | `SpotlightAction` | `packages/@openmaic/dsl/src/action.ts` [L30-L34](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L30-L34) |
| `laser` | `LaserAction` | `packages/@openmaic/dsl/src/action.ts` [L37-L41](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L37-L41) |

Both types include `elementId` targeting and configurable `dimOpacity` for spotlight effects.

### Synchronous Actions (Blocking Until Completion)

These **Action** types must complete before the lesson engine proceeds to the next action. They form the core of structured classroom interaction sequences.

| Action Type | Purpose | Interface Location |
|-------------|---------|------------------|
| `speech` | Text-to-speech narration with voice selection | [L46-L62](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L46-L62) |
| `wb_open` | Initialize whiteboard session | [L64-L66](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L64-L66) |
| `wb_draw_text` | Render text annotations | [L69-L80](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L69-L80) |
| `wb_draw_shape` | Draw geometric primitives | [L82-L92](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L82-L92) |
| `wb_draw_chart` | Render data visualizations | [L94-L108](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L94-L108) |
| `wb_draw_latex` | Typeset mathematical expressions | [L111-L121](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L111-L121) |
| `wb_draw_table` | Display tabular data | [L124-L134](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L124-L134) |
| `wb_draw_line` | Animate connector lines | [L136-L148](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L136-L148) |
| `wb_clear` | Erase whiteboard contents | [L150-L153](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L150-L153) |
| `wb_delete` | Remove specific elements | [L155-L159](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L155-L159) |
| `wb_close` | Terminate whiteboard session | [L161-L164](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L161-L164) |
| `wb_draw_code` | Display syntax-highlighted code blocks | [L166-L176](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L166-L176) |
| `wb_edit_code` | Enable live code editing | [L179-L186](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L179-L186) |
| `play_video` | Embedded video playback | [L188-L194](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L188-L194) |
| `discussion` | Facilitate student discussion prompts | [L196-L200](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L196-L200) |

### Widget Interaction Actions

These specialized **Action** types manipulate embedded interactive widgets, enabling dynamic state changes and visual feedback.

| Action Type | Function | Interface Location |
|-------------|----------|------------------|
| `widget_highlight` | Emphasize widget boundaries | [L204-L209](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L204-L209) |
| `widget_setState` | Update widget internal state | [L212-L217](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L212-L217) |
| `widget_annotation` | Attach contextual notes | [L219-L224](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L219-L224) |
| `widget_reveal` | Progressive disclosure of widget elements | [L226-L231](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts#L226-L231) |

## Working with Action Types in Code

All **Action** interfaces are accessible through the runtime shim at [`lib/types/action.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/types/action.ts). This thin re-export preserves existing import patterns while centralizing the canonical definitions from `@openmaic/dsl`.

### Import Pattern

```typescript
import { Action, ActionType, SpotlightAction, SpeechAction } from '@/lib/types/action';

```

The `Action` union type encompasses all 22 concrete interfaces, while `ActionType` provides the string literal union for type discrimination.

### Fire-and-Forget: Spotlight Action Example

```typescript
import { SpotlightAction } from '@/lib/types/action';

const spotlight: SpotlightAction = {
  id: 'a1',
  type: 'spotlight',
  elementId: 'slide-3',
  dimOpacity: 0.4,
};

```

The `id` field provides unique identification for action tracking; `elementId` targets specific DOM elements; `dimOpacity` controls background dimming intensity (0.0-1.0).

### Synchronous: Speech Action Example

```typescript
import { SpeechAction } from '@/lib/types/action';

const speech: SpeechAction = {
  id: 'a2',
  type: 'speech',
  text: 'Welcome to the lesson on photosynthesis.',
  voice: 'en-US-Wavenet-D',
  rate: 1.0,
  pitch: 0,
};

```

The `SpeechAction` interface [L46-L62] supports Google Cloud TTS voices, playback rate modification, and pitch adjustment. The runtime engine at [`lib/playback/action-navigation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/action-navigation.ts) blocks until audio completion or user skip.

### Whiteboard Drawing: Shape Action Example

```typescript
import { WbDrawShapeAction } from '@/lib/types/action';

const shape: WbDrawShapeAction = {
  id: 'a3',
  type: 'wb_draw_shape',
  shape: 'rectangle',
  x: 150,
  y: 100,
  width: 300,
  height: 150,
  fillColor: '#5b9bd5',
  strokeColor: '#2e75b5',
  strokeWidth: 2,
};

```

Supported `shape` values include `rectangle`, `ellipse`, `triangle`, and `arrow`. The coordinate system uses absolute positioning relative to the whiteboard canvas origin.

### Widget State Manipulation Example

```typescript
import { WidgetSetStateAction } from '@/lib/types/action';

const setState: WidgetSetStateAction = {
  id: 'a4',
  type: 'widget_setState',
  widgetId: 'simulator-thermo',
  state: { temperature: 37, pressure: 101.3 },
  transition: 'smooth',
};

```

The `state` property accepts any JSON-serializable object matching the target widget's state schema. The optional `transition` field enables animated state interpolation.

## Key Source Files for Action Development

| File Path | Responsibility |
|-----------|---------------|
| [[`lib/types/action.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/types/action.ts)](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/types/action.ts) | Runtime re-export shim maintaining backward compatibility |
| [`packages/@openmaic/dsl/src/action.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/packages/@openmaic/dsl/src/action.ts) | Canonical type definitions with execution category constants (`FIRE_AND_FORGET_ACTIONS`, `SYNC_ACTIONS`, `SLIDE_ONLY_ACTIONS`) |
| [[`lib/playback/action-navigation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/action-navigation.ts)](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/action-navigation.ts) | Playback engine consuming **Action** types during lesson execution |
| [[`components/workbench/chat/action-cluster.tsx`](https://github.com/THU-MAIC/OpenMAIC/blob/main/components/workbench/chat/action-cluster.tsx)](https://github.com/THU-MAIC/OpenMAIC/blob/main/components/workbench/chat/action-cluster.tsx) | UI component rendering action clusters in teacher chat panels |

## Execution Category Constants

The DSL package exports three helper constants for runtime classification:

- **`FIRE_AND_FORGET_ACTIONS`** — Array containing `'spotlight'`, `'laser'`
- **`SYNC_ACTIONS`** — Array of all 15 blocking action types
- **`SLIDE_ONLY_ACTIONS`** — Subset restricted to slide-level contexts

These constants enable pattern-matching in [`lib/playback/action-navigation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/action-navigation.ts) without hardcoding type strings throughout the codebase.

## Summary

- **OpenMAIC lib/action supports 22 classroom interaction action types** across three execution categories: fire-and-forget (2), synchronous (15), and widget interaction (4)
- **All types are defined in `@openmaic/dsl` and re-exported through [`lib/types/action.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/types/action.ts)** to maintain a single source of truth
- **Synchronous whiteboard actions** (`wb_*`) form the largest category, enabling comprehensive annotation and drawing capabilities
- **Widget interaction actions** provide stateful manipulation of embedded interactive components
- **Execution category constants** (`FIRE_AND_FORGET_ACTIONS`, `SYNC_ACTIONS`) facilitate runtime behavior discrimination in the playback engine

## Frequently Asked Questions

### How do I determine if an action blocks lesson progression?

Check the `SYNC_ACTIONS` constant exported from `@openmaic/dsl`. Actions in this array—including all `wb_*`, `speech`, `play_video`, and `discussion` types—must complete before the playback engine advances. Fire-and-forget actions (`spotlight`, `laser`) execute in parallel.

### Can I extend lib/action with custom action types?

The **Action** union in `packages/@openmaic/dsl/src/action.ts` is the extension point. Add your interface definition there, update the union type, and modify [`lib/playback/action-navigation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/action-navigation.ts) to handle execution. Re-export through [`lib/types/action.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/types/action.ts) to maintain import consistency.

### What distinguishes widget_setState from widget_highlight?

**`widget_setState`** modifies internal widget state (data model, configuration) and requires the widget to implement state handlers. **`widget_highlight`** applies transient visual effects (border glow, pulse animation) without changing widget internals—purely presentational feedback for classroom attention direction.

### Where does the runtime enforce action type discrimination?

The playback engine at [`lib/playback/action-navigation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/playback/action-navigation.ts) consumes **Action** unions and dispatches to specialized handlers based on `action.type`. TypeScript's exhaustiveness checking ensures all 22 action types have corresponding implementation branches, preventing unhandled action failures at runtime.