# How the OpenMAIC Action Engine Handles Synchronous Speech Actions

> Discover how the OpenMAIC Action Engine manages synchronous speech actions by returning Promises that resolve after audio playback, ensuring smooth sequential execution.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: internals
- Published: 2026-09-03

---

**The OpenMAIC `ActionEngine` treats speech as a synchronous action by returning a `Promise` that resolves only after audio playback completes, ensuring sequential execution of subsequent actions.**

The `ActionEngine` serves as the central execution coordinator for all agent behaviors in the THU-MAIC/OpenMAIC framework. Unlike fire-and-forget actions such as spotlights or lasers, speech actions require precise synchronization to guarantee that narration finishes before subsequent visual cues appear. This article examines the internal mechanics of how the engine processes these synchronous speech actions according to the source code implementation.

## Action Classification in the ActionEngine

The `ActionEngine` categorizes all incoming actions into two distinct execution models:

- **Fire-and-forget** – Actions like `spotlight` or `laser` dispatch immediately and return control to the caller without waiting for completion.
- **Synchronous** – Actions like `speech`, `play_video`, and whiteboard drawing operations return a `Promise` that settles only after the action fully completes.

This architectural distinction ensures that time-sensitive sequences—such as highlighting an element only after the narrator finishes speaking—execute in the correct order without race conditions.

## The Speech Action Execution Flow

When the engine receives a speech action, it follows a rigorous four-step process to guarantee synchronous completion tracking.

### Routing and Type Detection

In [`lib/action/engine.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/engine.ts), the `execute()` method (lines [32‑47](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/engine.ts#L32-L47)) inspects the incoming action type and routes speech payloads to the dedicated handler:

```typescript
// Simplified routing logic from engine.ts
switch (action.type) {
  case 'speech':
    return this.executeSpeech(action);
  // ... other cases
}

```

The method returns the result of `executeSpeech()` directly, propagating the promise up to the caller.

### Audio Player Validation

The `executeSpeech()` method first verifies that an `AudioPlayer` instance is available before attempting playback. If the audio infrastructure is missing, the method returns immediately to prevent runtime errors:

```typescript
if (!this.audioPlayer) return;

```

### Promise-Based Completion Handling

The core synchronization mechanism relies on JavaScript promises coupled with event listeners. The engine constructs a new `Promise` that resolves through an `onEnded` callback registered on the `AudioPlayer` instance:

```typescript
// From lib/action/engine.ts around line 40
audioPlayer.onEnded(() => {
  resolve(); // Resolves only when audio finishes naturally
});

```

The engine then initiates playback using the speech action’s `audioId` (or legacy `audioUrl`). If playback fails to start, the promise resolves immediately (lines [44‑47](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/engine.ts#L44-L47)) to prevent indefinite blocking:

```typescript
const success = audioPlayer.play(action.audioId);
if (!success) {
  resolve(); // Immediate resolution on failure
}

```

### Return Value and Caller Awaiting

The method returns a `Promise<void>` that callers can `await`. This design ensures that any code following a speech action—such as triggering a spotlight—waits for the audio duration to elapse before executing.

## Practical Implementation Examples

The following patterns demonstrate how to leverage the synchronous nature of speech actions in your OpenMAIC applications.

### Basic Speech Execution

This example initializes the engine with an `AudioPlayer` and awaits speech completion:

```typescript
import { ActionEngine } from '@/lib/action/engine';
import { AudioPlayer } from '@/lib/utils/audio-player';

const audioPlayer = new AudioPlayer();
const engine = new ActionEngine(stageStore, audioPlayer);

const speechAction = {
  id: 'speech-1',
  type: 'speech',
  text: 'Welcome to the lesson!',
  audioId: 'audio-123',  // Pre-generated audio asset identifier
};

// Blocks until audio finishes playing
await engine.execute(speechAction);
console.log('Speech completed');

```

### Chaining Dependent Actions

Because speech actions return promises, you can sequence visual cues to appear only after narration completes:

```typescript
// Spotlight appears only after narration finishes
await engine.execute(speechAction);
engine.execute({ 
  id: 'spot-1', 
  type: 'spotlight', 
  elementId: 'elem-42' 
});

```

This pattern eliminates timing bugs where visual elements might appear before the corresponding audio explanation begins.

## Key Source Files and Components

Understanding these three files provides complete visibility into the speech synchronization architecture:

- **[`lib/action/engine.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/action/engine.ts)** – Contains the `ActionEngine` class with the `execute()` routing logic and `executeSpeech()` method that implements the promise-based completion tracking.
- **[`lib/types/action.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/types/action.ts)** – Defines the `SpeechAction` interface and type definitions that enforce the contract for speech payloads including `audioId` and `text` properties.
- **[`lib/utils/audio-player.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/utils/audio-player.ts)** – Provides the audio abstraction layer that emits `ended` events, enabling the engine to detect when playback concludes.

## Summary

- The **ActionEngine** classifies actions as either fire-and-forget or synchronous, with speech falling into the latter category.
- **Synchronous speech actions** return a `Promise<void>` that resolves via an `onEnded` callback registered on the `AudioPlayer` instance.
- The **routing logic** in `execute()` (lines 32‑47) forwards speech actions to `executeSpeech()`, which validates the audio player availability before initiating playback.
- **Error handling** ensures promises resolve immediately if playback fails to start, preventing application hangs.
- This architecture guarantees **sequential action execution**, allowing subsequent actions to wait for speech completion naturally using standard `await` semantics.

## Frequently Asked Questions

### What makes speech actions synchronous in OpenMAIC?

Speech actions are synchronous because the `ActionEngine` returns an active `Promise` rather than completing immediately. This promise remains pending until the `AudioPlayer` emits an `ended` event or encounters an error, allowing the JavaScript event loop to pause execution until the audio finishes.

### How does the Action Engine sequence actions after speech?

Because `engine.execute()` returns a promise for speech actions, developers can use standard `await` syntax or `.then()` chains. The engine will not process subsequent actions in the queue until the previous speech promise resolves, creating an implicit barrier that preserves narrative flow.

### What happens if the AudioPlayer fails to start playback?

If `audioPlayer.play()` returns a falsy value indicating startup failure, the `executeSpeech()` method immediately calls `resolve()` (lines 44‑47). This prevents the promise from remaining pending indefinitely while still allowing the action sequence to continue, treating the failure as a completed (though unsuccessful) operation.

### Where are the speech action types defined?

The TypeScript interfaces for speech actions, including the `SpeechAction` type with its `audioId` and `text` properties, are defined in [`lib/types/action.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/types/action.ts). These type definitions ensure compile-time safety when constructing action payloads for the engine.