How OpenMAIC Generates Lesson Content: The Complete Technical Pipeline

OpenMAIC generates lesson content by transforming user requirements into persisted scenes through an AI-orchestrated pipeline that validates input, generates visual elements and playback actions, synthesizes narration, and maintains recoverable checkpoints before redirecting to the classroom view.

The THU-MAIC/OpenMAIC repository implements a sophisticated lesson generation system that converts high-level educational requirements into interactive classroom experiences. Understanding the process for generating lesson content in OpenMAIC requires examining the coordinated interaction between agent runtime tools, AI generation engines, and persistent storage layers orchestrated by the generation AI call factory (createGenerationAiCallFactory).

The 8-Step Lesson Generation Pipeline

The generation process follows a strict sequence from requirement capture to classroom delivery:

1. Receive Requirements A client (UI or API) supplies a requirements object containing the topic, language, and optional style brief. The system stores this request in sessionStorage as generationSession, as demonstrated in the e2e test generation-flow.spec.ts.

2. Outline Review (Optional) When the "always review" setting is enabled, the GenerationPreviewPage displays the outline editor via the reviewOutlineButton and editorTitle elements, allowing users to modify the lesson structure before content generation begins.

3. Generate Scene Content For each scene, the generate_scene tool validates input, builds a SceneOutline, and calls the generation AI. The implementation in lib/server/agent-runtime/generation-tools.ts (specifically the generateScene function) assembles scene content including elements, background, and media assets.

4. Convert Runtime to Generation Shape The toGenerationContent function in lib/server/agent-runtime/generation-content.ts adapts persisted SceneContent to the shape required by AI-driven generation functions. This conversion handles the translation of runtime canvas elements to generation-compatible formats.

5. Generate Actions After visual content generation, the generate_actions tool (implemented as generateActionsTool in generation-tools.ts) calls the AI to create playback actions including speech and animations. The system filters out unknown action types before persistence.

6. Synthesize Narration (Optional) If synthesizeAudio is not disabled, the synthesizeSceneNarration function executes text-to-speech (TTS) processing on generated speech actions via the synthesizeTts dependency, persisting audio files for classroom playback.

7. Persist and Checkpoint Every successful write creates a checkpoint through deps.onCheckpoint within generation-tools.ts. The putSceneBringingCurrent function handles persistence while reporting unresolved media placeholders for recovery or audit purposes.

8. Redirect to Classroom Once all scenes generate successfully or the user confirms the outline, the UI redirects to the classroom view. The e2e flow confirms this via GenerationPreviewPage.waitForRedirectToClassroom().

Core Data Flow Architecture

The architectural contract for lesson generation follows five distinct phases:

• User → UI: Stores a JSON session with requirements in browser storage
• UI → Agent Tools: Invokes generate_scene, generate_actions, and duplicate_scene tools
• Agent Tools → Generation Engine: The @openmaic/generation package produces GeneratedSlideContent, GeneratedQuizContent, and other specialized content types
• Engine → Persistence: Utilizes store.saveDocument and putSceneBringingCurrent for durable storage
• Persistence → UI: Final delivery to the classroom preview for immediate playback

Implementation Examples

Initialize a Generation Session

The client stores requirements in session storage before triggering the pipeline:

const GENERATION_SESSION = JSON.stringify({
  sessionId: 'e2e-test-session',
  requirements: { requirement: '讲解光合作用', language: 'zh-CN' },
  pdfText: '',
  pdfImages: [],
  imageStorageIds: [],
  sceneOutlines: null,
  currentStep: 'generating',
});

await page.addInitScript(
  ({ session }) => sessionStorage.setItem('generationSession', session),
  { session: GENERATION_SESSION },
);

Generate a Scene via Agent Tools

Call the generate_scene tool with structured parameters:

await mockApi.setupGenerationMocks();

const params = {
  stageId: 'stage-123',
  order: 1,
  title: '光合作用概述',
  type: 'slide',
  brief: '介绍光合作用的基本概念',
  media: [
    { src: 'https://example.com/leaf.png', description: '叶绿体示意图' },
  ],
};

const result = await agentToolCall('generate_scene', params);
console.log(result.content[0].text); // "Page 1 "光合作用概述" persisted."

Regenerate Actions for Existing Content

Update playback actions for a persisted scene:

const actionsResult = await agentToolCall('generate_actions', {
  stageId: 'stage-123',
  sceneId: 'scene-p1',
  synthesizeAudio: true,
});

console.log(actionsResult.details.actions.length, 'actions persisted');

Duplicate Scenes for Templating

Reuse existing content structures:

const dupResult = await agentToolCall('duplicate_scene', {
  stageId: 'stage-123',
  templateOrder: 2,
  targetOrder: 5,
  title: '光合作用实验',
});

console.log(dupResult.content[0].text); // "Duplicated "原始标题" at order 5."

Navigate to Classroom Playback

Complete the generation flow via UI navigation:

await page.goto('/generation-preview');
await page.getByRole('button', { name: /confirm and generate course/i }).click();
await page.waitForURL(/\/classroom\//);

Key Source Files and Functions

File	Role
lib/server/agent-runtime/generation-tools.ts	Implements the agent-tool suite (generate_scene, list_scenes, generate_actions, duplicate_scene). Handles validation, AI calls, persistence, and checkpointing via putSceneBringingCurrent.
lib/server/agent-runtime/generation-content.ts	Adapts persisted SceneContent to generation-time shapes through toGenerationContent.
e2e/pages/generation-preview.page.ts	Playwright page object driving UI flow: preview, outline review, and classroom redirection.
e2e/tests/generation-flow.spec.ts	End-to-end test stitching together the complete generation pipeline.
scripts/generation-node-smoke.mjs	CLI smoke test for verifying generation tools outside browser contexts.
lib/types/generation.ts	Type definitions for Generated*Content shapes returned by the generation engine.

Summary

• OpenMAIC implements an 8-step pipeline for generating lesson content, from requirement capture through classroom delivery.
• The generate_scene and generate_actions tools in generation-tools.ts serve as the primary interfaces between the UI and AI generation engine.
• Content transformation occurs through toGenerationContent to bridge runtime storage formats and generation requirements.
• Checkpointing via deps.onCheckpoint ensures recoverability and auditability throughout the generation process.
• Optional TTS synthesis via synthesizeSceneNarration creates audio narration for interactive playback.

Frequently Asked Questions

What triggers the lesson generation process in OpenMAIC?

The process initiates when a client stores a generationSession object in sessionStorage containing requirements (topic, language, style) and navigates to the generation preview page. The GenerationPreviewPage then orchestrates the workflow, optionally presenting an outline editor before invoking the generate_scene tool for each lesson segment.

How does OpenMAIC handle audio narration for generated lessons?

When the synthesizeAudio parameter is enabled (default behavior unless explicitly disabled), the system calls synthesizeSceneNarration after generating actions. This function processes speech actions through the synthesizeTts dependency to generate audio files, which are persisted alongside the scene content for synchronous playback in the classroom view.

What happens if the generation process is interrupted?

The checkpointing mechanism in lib/server/agent-runtime/generation-tools.ts creates a recoverable state after every successful write via deps.onCheckpoint. If interruption occurs, unresolved media placeholders are reported, and the system can resume from the last valid checkpoint without regenerating completed scenes, maintaining data integrity through putSceneBringingCurrent.

Can developers customize the AI model used for content generation?

Yes. The entire pipeline abstracts the actual LLM or inference service through the generation AI call factory (createGenerationAiCallFactory). Developers can inject custom implementations of this factory to replace the default generation backend while maintaining compatibility with the existing tool definitions in generation-tools.ts and content adapters in generation-content.ts.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}