# How OpenMAIC Generates Lesson Content: The Complete Technical Pipeline

> Explore the OpenMAIC technical pipeline for lesson content generation. Learn how user needs become validated scenes with AI-driven visuals, narration, and checkpoints.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: deep-dive
- Published: 2026-08-30

---

**OpenMAIC generates lesson content by transforming user requirements into persisted scenes through an AI-orchestrated pipeline that validates input, generates visual elements and playback actions, synthesizes narration, and maintains recoverable checkpoints before redirecting to the classroom view.**

The THU-MAIC/OpenMAIC repository implements a sophisticated lesson generation system that converts high-level educational requirements into interactive classroom experiences. Understanding the process for generating lesson content in OpenMAIC requires examining the coordinated interaction between agent runtime tools, AI generation engines, and persistent storage layers orchestrated by the **generation AI call factory** (`createGenerationAiCallFactory`).

## The 8-Step Lesson Generation Pipeline

The generation process follows a strict sequence from requirement capture to classroom delivery:

**1. Receive Requirements**
A client (UI or API) supplies a `requirements` object containing the topic, language, and optional style brief. The system stores this request in `sessionStorage` as `generationSession`, as demonstrated in the e2e test [`generation-flow.spec.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/generation-flow.spec.ts).

**2. Outline Review (Optional)**
When the "always review" setting is enabled, the `GenerationPreviewPage` displays the outline editor via the `reviewOutlineButton` and `editorTitle` elements, allowing users to modify the lesson structure before content generation begins.

**3. Generate Scene Content**
For each scene, the `generate_scene` tool validates input, builds a **SceneOutline**, and calls the generation AI. The implementation in [`lib/server/agent-runtime/generation-tools.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/agent-runtime/generation-tools.ts) (specifically the `generateScene` function) assembles scene content including elements, background, and media assets.

**4. Convert Runtime to Generation Shape**
The `toGenerationContent` function in [`lib/server/agent-runtime/generation-content.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/agent-runtime/generation-content.ts) adapts persisted `SceneContent` to the shape required by AI-driven generation functions. This conversion handles the translation of runtime canvas elements to generation-compatible formats.

**5. Generate Actions**
After visual content generation, the `generate_actions` tool (implemented as `generateActionsTool` in [`generation-tools.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/generation-tools.ts)) calls the AI to create playback actions including speech and animations. The system filters out unknown action types before persistence.

**6. Synthesize Narration (Optional)**
If `synthesizeAudio` is not disabled, the `synthesizeSceneNarration` function executes text-to-speech (TTS) processing on generated speech actions via the `synthesizeTts` dependency, persisting audio files for classroom playback.

**7. Persist and Checkpoint**
Every successful write creates a checkpoint through `deps.onCheckpoint` within [`generation-tools.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/generation-tools.ts). The `putSceneBringingCurrent` function handles persistence while reporting unresolved media placeholders for recovery or audit purposes.

**8. Redirect to Classroom**
Once all scenes generate successfully or the user confirms the outline, the UI redirects to the classroom view. The e2e flow confirms this via `GenerationPreviewPage.waitForRedirectToClassroom()`.

## Core Data Flow Architecture

The architectural contract for lesson generation follows five distinct phases:

- **User → UI**: Stores a JSON session with requirements in browser storage
- **UI → Agent Tools**: Invokes `generate_scene`, `generate_actions`, and `duplicate_scene` tools
- **Agent Tools → Generation Engine**: The `@openmaic/generation` package produces `GeneratedSlideContent`, `GeneratedQuizContent`, and other specialized content types
- **Engine → Persistence**: Utilizes `store.saveDocument` and `putSceneBringingCurrent` for durable storage
- **Persistence → UI**: Final delivery to the classroom preview for immediate playback

## Implementation Examples

### Initialize a Generation Session

The client stores requirements in session storage before triggering the pipeline:

```typescript
const GENERATION_SESSION = JSON.stringify({
  sessionId: 'e2e-test-session',
  requirements: { requirement: '讲解光合作用', language: 'zh-CN' },
  pdfText: '',
  pdfImages: [],
  imageStorageIds: [],
  sceneOutlines: null,
  currentStep: 'generating',
});

await page.addInitScript(
  ({ session }) => sessionStorage.setItem('generationSession', session),
  { session: GENERATION_SESSION },
);

```

### Generate a Scene via Agent Tools

Call the `generate_scene` tool with structured parameters:

```typescript
await mockApi.setupGenerationMocks();

const params = {
  stageId: 'stage-123',
  order: 1,
  title: '光合作用概述',
  type: 'slide',
  brief: '介绍光合作用的基本概念',
  media: [
    { src: 'https://example.com/leaf.png', description: '叶绿体示意图' },
  ],
};

const result = await agentToolCall('generate_scene', params);
console.log(result.content[0].text); // "Page 1 "光合作用概述" persisted."

```

### Regenerate Actions for Existing Content

Update playback actions for a persisted scene:

```typescript
const actionsResult = await agentToolCall('generate_actions', {
  stageId: 'stage-123',
  sceneId: 'scene-p1',
  synthesizeAudio: true,
});

console.log(actionsResult.details.actions.length, 'actions persisted');

```

### Duplicate Scenes for Templating

Reuse existing content structures:

```typescript
const dupResult = await agentToolCall('duplicate_scene', {
  stageId: 'stage-123',
  templateOrder: 2,
  targetOrder: 5,
  title: '光合作用实验',
});

console.log(dupResult.content[0].text); // "Duplicated "原始标题" at order 5."

```

### Navigate to Classroom Playback

Complete the generation flow via UI navigation:

```typescript
await page.goto('/generation-preview');
await page.getByRole('button', { name: /confirm and generate course/i }).click();
await page.waitForURL(/\/classroom\//);

```

## Key Source Files and Functions

| File | Role |
|------|------|
| [`lib/server/agent-runtime/generation-tools.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/agent-runtime/generation-tools.ts) | Implements the agent-tool suite (`generate_scene`, `list_scenes`, `generate_actions`, `duplicate_scene`). Handles validation, AI calls, persistence, and checkpointing via `putSceneBringingCurrent`. |
| [`lib/server/agent-runtime/generation-content.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/agent-runtime/generation-content.ts) | Adapts persisted `SceneContent` to generation-time shapes through `toGenerationContent`. |
| [`e2e/pages/generation-preview.page.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/e2e/pages/generation-preview.page.ts) | Playwright page object driving UI flow: preview, outline review, and classroom redirection. |
| [`e2e/tests/generation-flow.spec.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/e2e/tests/generation-flow.spec.ts) | End-to-end test stitching together the complete generation pipeline. |
| `scripts/generation-node-smoke.mjs` | CLI smoke test for verifying generation tools outside browser contexts. |
| [`lib/types/generation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/types/generation.ts) | Type definitions for `Generated*Content` shapes returned by the generation engine. |

## Summary

- OpenMAIC implements an 8-step pipeline for generating lesson content, from requirement capture through classroom delivery.
- The `generate_scene` and `generate_actions` tools in [`generation-tools.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/generation-tools.ts) serve as the primary interfaces between the UI and AI generation engine.
- Content transformation occurs through `toGenerationContent` to bridge runtime storage formats and generation requirements.
- Checkpointing via `deps.onCheckpoint` ensures recoverability and auditability throughout the generation process.
- Optional TTS synthesis via `synthesizeSceneNarration` creates audio narration for interactive playback.

## Frequently Asked Questions

### What triggers the lesson generation process in OpenMAIC?

The process initiates when a client stores a `generationSession` object in `sessionStorage` containing requirements (topic, language, style) and navigates to the generation preview page. The `GenerationPreviewPage` then orchestrates the workflow, optionally presenting an outline editor before invoking the `generate_scene` tool for each lesson segment.

### How does OpenMAIC handle audio narration for generated lessons?

When the `synthesizeAudio` parameter is enabled (default behavior unless explicitly disabled), the system calls `synthesizeSceneNarration` after generating actions. This function processes speech actions through the `synthesizeTts` dependency to generate audio files, which are persisted alongside the scene content for synchronous playback in the classroom view.

### What happens if the generation process is interrupted?

The checkpointing mechanism in [`lib/server/agent-runtime/generation-tools.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/agent-runtime/generation-tools.ts) creates a recoverable state after every successful write via `deps.onCheckpoint`. If interruption occurs, unresolved media placeholders are reported, and the system can resume from the last valid checkpoint without regenerating completed scenes, maintaining data integrity through `putSceneBringingCurrent`.

### Can developers customize the AI model used for content generation?

Yes. The entire pipeline abstracts the actual LLM or inference service through the **generation AI call factory** (`createGenerationAiCallFactory`). Developers can inject custom implementations of this factory to replace the default generation backend while maintaining compatibility with the existing tool definitions in [`generation-tools.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/generation-tools.ts) and content adapters in [`generation-content.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/generation-content.ts).