Where to Find the Scene Content Generation Code in OpenMAIC: Complete Architecture Guide

The scene content generation code in OpenMAIC is located in app/api/generate/scene-content/route.ts, which delegates LLM processing to the generateSceneContent function from the workspace package @openmaic/generation.

OpenMAIC is an open-source AI-powered courseware generation system developed by THU-MAIC. Understanding where and how scene content generation happens is essential for developers who want to customize prompts, add new slide types, or integrate additional LLM providers. This guide walks through the exact file locations, data flow, and key implementation details based on the OpenMAIC source code.

Entry Point: The Scene Content API Route

The HTTP handler for scene content generation lives at app/api/generate/scene-content/route.ts. This file receives POST requests from the frontend, validates the payload, and orchestrates the generation pipeline.

According to the OpenMAIC source, the route performs these steps:

• Validates the incoming JSON payload against expected schema
• Resolves the LLM model using resolveModelFromRequest (supports scene-content or scene-content:<type> variants)
• Prepares vision inputs by resolving image assets to byte data when using vision-capable models
• Invokes the generation engine via import { generateSceneContent } from '@openmaic/generation'
• Returns { content, effectiveOutline } wrapped in a standard API response

The route deliberately separates scene-content (text generation) from scene-actions (interactive elements), which has its own endpoint at /api/generate/scene-actions.

Core Generation Engine: @openmaic/generation

The actual prompt construction and LLM invocation happens inside the workspace package @openmaic/generation. The generateSceneContent function exported from this package handles:

• Prompt template selection based on outline type (slide, quiz, interactive, pbl)
• Multimodal prompt building for vision-enabled models
• LLM API orchestration with streaming and error handling
• Post-processing of generated content

This architecture keeps the API route thin and moves reusable generation logic into a shared package that other routes can import.

Vision and Media Handling Pipeline

OpenMAIC supports vision-capable LLMs for analyzing slide images and PDF extracts. The relevant source files are:

File	Purpose
lib/persistence/resolve-vision-images.ts	Converts image IDs to byte arrays for LLM consumption
lib/document/bundle.ts	Provides sortDocumentImagesForVision to order images contextually
lib/server/resolve-model.ts	Determines model selection from request headers and config

When modelInfo?.capabilities?.vision is detected, the route calls resolveVisionImagesForPrompt and buildVisionUserContent to construct multimodal messages. For non-vision models, the system falls back to text-only prompts with image descriptions.

Outline Processing and Fallback Logic

The route applies intelligent defaults through applyOutlineFallbacks. This function handles:

• Procedural skill detection — enabling step-by-step generation patterns
• Language model awareness — adjusting for model-specific capabilities
• Content type specialization — applying different defaults for slides versus quizzes versus PBL pages

The effectiveOutline returned in the response reflects these applied transformations, letting the frontend know exactly what parameters were used.

Media Placeholder System

scene content generation produces placeholder identifiers rather than final assets. The route generates IDs like gen_img_1 and gen_vid_1 that the client later replaces with actual generated or selected media. This decouples text generation from expensive image/video synthesis.

Error Handling and Response Format

All errors are caught and normalized through helpers in lib/server/api-response.ts:

• LLM failures → llmApiError with retry guidance
• Validation failures → apiError with field-level details
• Success → apiSuccess wrapping { content, effectiveOutline }

Example API Call

// POST /api/generate/scene-content
const payload = {
  outline: {
    title: "Introduction to Fractions",
    type: "slide",
    content: "Explain fractions using visual examples"
  },
  allOutlines: [/* full course structure */],
  pdfImages: [{ id: "img-01", src: "data:image/png;base64,..." }],
  imageMapping: { "img-01": "asset-id-1234" },
  stageId: "stage-xyz",
  agents: [{ name: "teacher", role: "explain" }],
  languageDirective: "Use simple Chinese",
  requirements: { proceduralSkill: true }
};

fetch("/api/generate/scene-content", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "x-user-locale": "zh-CN"
  },
  body: JSON.stringify(payload)
});

Response:

{
  "success": true,
  "data": {
    "content": "# Introduction to Fractions\n\nA fraction represents...",

    "effectiveOutline": { /* normalized outline with fallbacks applied */ }
  }
}

Key Source File Reference

Path	GitHub URL
app/api/generate/scene-content/route.ts	View on GitHub
packages/@openmaic/generation	View on GitHub
lib/persistence/resolve-vision-images.ts	View on GitHub
lib/document/bundle.ts	View on GitHub
lib/server/resolve-model.ts	View on GitHub
lib/server/api-response.ts	View on GitHub

Summary

• Primary entry point: app/api/generate/scene-content/route.ts handles HTTP requests and orchestration
• Core generation logic: generateSceneContent from @openmaic/generation workspace package
• Vision support: Enabled through lib/persistence/resolve-vision-images.ts and lib/document/bundle.ts
• Model resolution: Configured in lib/server/resolve-model.ts
• Response standardization: Provided by lib/server/api-response.ts
• Architecture pattern: Route delegates to workspace package; media placeholders resolved client-side

Frequently Asked Questions

What is the difference between scene-content and scene-actions generation?

The scene-content endpoint generates textual material for slides, quizzes, interactive elements, or PBL pages. The separate scene-actions endpoint (at /api/generate/scene-actions) generates the interactive behaviors that follow the content—such as button clicks, form submissions, or adaptive feedback. This split allows the system to cache or regenerate content independently from interactions.

How does OpenMAIC handle images in scene content generation?

When the resolved model has capabilities.vision enabled, OpenMAIC resolves image IDs to byte data via resolveVisionImagesForPrompt, sorts them contextually with sortDocumentImagesForVision, and builds multimodal prompts using buildVisionUserContent. For non-vision models, the system falls back to text descriptions or omits visual references entirely.

Can I customize the prompt templates for scene content generation?

Yes, but the modification point depends on your goal. To change prompt structure, examine the @openmaic/generation workspace package where generateSceneContent constructs templates. To adjust how outlines are pre-processed before generation, modify applyOutlineFallbacks in the route file or the outline transformation utilities it calls.

What model identifiers can I use for scene-content generation?

The system accepts scene-content as the base identifier, or scene-content:<type> for type-specific routing (e.g., scene-content:quiz). The actual LLM selection happens in resolveModelFromRequest in lib/server/resolve-model.ts, which maps these logical identifiers to configured provider models based on headers and deployment configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}