# How the `generateSceneContent` Function Works in OpenMAIC: Deep Dive into Lesson Content Generation

> Explore the `generateSceneContent` function in OpenMAIC. This deep dive reveals how it dispatches scene outlines to specialized generators for slides, quizzes, widgets, and PBL lessons via a single async API.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: deep-dive
- Published: 2026-09-06

---

**The `generateSceneContent` function in OpenMAIC acts as a unified dispatcher that routes scene outlines to specialized generators based on type, handling slides, quizzes, interactive widgets, and PBL lessons through a single async API.**

`generateSceneContent` serves as the core entry point of the **@openmaic/generation** package. Located in `packages/@openmaic/generation/src/scene-generator.ts`, it transforms high-level **SceneOutline** objects into fully rendered lesson content by delegating to type-specific generators. This architecture enables consistent content generation across multiple interfaces—the server-side route `/api/scene-generation`, the React hook `useSceneGenerator`, and the CLI script `scripts/generation-node-smoke.mjs`.

## Function Signature and Core Responsibilities

The function accepts three parameters that define its flexible contract:

- **`outline`** – A `SceneOutline` describing the target output (slide, quiz, interactive widget, or PBL lesson)
- **`aiCall`** – An async function that interfaces with the configured LLM
- **`options`** – Optional configuration controlling image handling, language directives, procedural skills, and logging

```typescript
// Simplified signature from scene-generator.ts
async function generateSceneContent(
  outline: SceneOutline,
  aiCall: AICallFunction,
  options?: GenerationOptions
): Promise<GeneratedContent | null>

```

The implementation demonstrates clean separation of concerns: common setup, type-based routing, and specialized generation pipelines.

## Common Setup and Configuration Phase

Before any generation begins, the function establishes its execution context. Lines 27-52 in [`scene-generator.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/scene-generator.ts) handle this initialization:

```typescript
// From packages/@openmaic/generation/src/scene-generator.ts#L27-L52
const logger = options.logger ?? noopGenerationLogger;
const {
  imageMapping,
  generatedMediaMapping,
  visionEnabled,
  languageDirectives,
  allowProceduralSkill,
  // ... additional destructured options
} = options ?? {};

```

This phase ensures that all downstream generators have consistent access to logging, media resolution maps, and feature flags. The **nullish coalescing** pattern (`??`) guarantees safe defaults without polluting call sites.

## Interactive Widget Generation Pipeline

When `outline.type === 'interactive'`, the function executes a dedicated migration and dispatch path (lines 54-80).

### Legacy Configuration Migration

Older outlines containing `interactiveConfig` objects are automatically upgraded:

```typescript
// From packages/@openmaic/generation/src/scene-generator.ts#L54-L80
if (outline.interactiveConfig) {
  outline = convertInteractiveConfigToWidget(outline);
}

```

The `convertInteractiveConfigToWidget` helper bridges API evolution without breaking existing consumers.

### Widget Type Resolution and Fallback

If the migrated outline still lacks a `widgetType`, the system applies **backward-compatible defaults**:

- Missing `widgetType` → defaults to **'simulation'**

This ensures that legacy content continues to render predictably.

### Delegation to `generateWidgetContent`

The final step routes to `generateWidgetContent`, which handles all five supported widget types:

| Widget Type | Description |
|-------------|-------------|
| `simulation` | Interactive physics or system simulations |
| `diagram` | Editable or explorable diagrams |
| `code` | Executable code playgrounds |
| `game` | Educational mini-games |
| `visualization3d` | Three-dimensional data or model viewers |

```typescript
// Widget generation delegation
return generateWidgetContent(outline, aiCall, {
  logger,
  visionEnabled,
  languageDirectives,
});

```

## Non-Interactive Content Generation

For non-interactive outlines, a **switch statement** on `outline.type` determines the appropriate generator (lines 82-110).

### Slide Generation via `generateSlideContent`

**`generateSlideContent`** handles the most complex non-interactive path. According to the source, it manages:

- **Image insertion and resolution** – mapping placeholder IDs to actual URLs
- **Video embedding** – with normalized references
- **LaTeX rendering** – converting mathematical expressions to HTML
- **Element defaults** – applying DSL normalization and aspect ratio corrections

Key helper functions support this pipeline:

- `resolveImageIds` – replaces `img_1`, `img_2` placeholders with mapped URLs
- `normalizeGeneratedVideoRefs` – sanitizes video reference structures
- `processLatexElements` – renders LaTeX via KaTeX with malformed-element removal
- `fixElementDefaults` – reconciles generated elements against PDF asset dimensions

```typescript
// Simplified slide generation flow
case 'slide':
  const slideContent = await generateSlideContent(outline, aiCall, options);
  return fixElementDefaults(
    resolveImageIds(slideContent, imageMapping),
    generatedMediaMapping
  );

```

### Quiz Generation via `generateQuizContent`

Quiz outlines route to **`generateQuizContent`**, which produces structured question-answer objects. This generator focuses on pedagogical correctness and answer key accuracy rather than media handling.

### PBL Lesson Generation via `generatePBLSceneContent`

**Problem-Based Learning (PBL)** scenes follow a more complex orchestration:

```typescript
// From packages/@openmaic/generation/src/scene-generator.ts#L100-L110
case 'pbl':
  return generatePBLSceneContent(outline, aiCall, {
    logger,
    allowProceduralSkill,
    // PBL-specific options
  });

```

The PBL generator optionally falls back to a **loop-based planning mode** when initial generation requires refinement, enabling iterative lesson structure development.

### Unknown Type Handling

Unrecognized outline types return `null`, signaling to callers that no content could be generated. This explicit failure mode supports graceful degradation in production pipelines.

## Media Resolution and Post-Processing Architecture

Several internal functions standardize content across all generation paths:

### `resolveImageIds` and `normalizeGeneratedVideoRefs`

These utilities bridge the gap between LLM-generated placeholders and actual media assets:

```typescript
// Conceptual usage across generators
const resolvedContent = resolveImageIds(
  rawGeneratedContent,
  imageMapping  // Maps "img_1" → "https://cdn.example.com/actual-image.png"
);

```

### `fixElementDefaults`

This helper enforces **DSL consistency** by:

1. Calling `normalizeElement` for each generated element
2. Adjusting image aspect ratios to match actual PDF-derived assets
3. Ensuring coordinate and sizing defaults align with rendering requirements

### `processLatexElements`

Mathematical content undergoes server-side rendering:

- KaTeX converts LaTeX strings to performant HTML
- Malformed or unparseable expressions are removed to prevent client-side crashes

## Practical Usage Examples

### Node.js Script Direct Execution

```typescript
// scripts/generation-node-smoke.mjs
import { generateSceneContent } from '@openmaic/generation';
import { slideOutline } from './fixtures/slide-outline';
import { aiCall } from './ai-call-mock';

async function demo() {
  const content = await generateSceneContent(
    slideOutline(),
    aiCall,
    {
      logger: console,
      visionEnabled: false,
      allowProceduralSkill: true,
    },
  );
  console.log(content);
}
demo();

```

This pattern supports **CI/CD smoke testing** and **batch content generation** workflows.

### Server-Side API Route Integration

```typescript
// lib/server/scene-generation.ts
export async function sceneGenerationHandler(req, res) {
  const { outline } = req.body;
  const aiCall = makeAICall(req);
  
  const content = await generateSceneContent(outline, aiCall, {
    logger: requestLogger,
    imageMapping: req.imageMapping,
  });
  
  res.json({ content });
}

```

The server context propagates request-specific media mappings and authentication-bound AI callers.

### React Hook Abstraction

```tsx
// hooks/use-scene-generator.ts
import { useCallback } from 'react';
import { generateSceneContent } from '@openmaic/generation';
import { useAiCall } from '../hooks/useAiCall';

export function useSceneGenerator() {
  const aiCall = useAiCall();
  
  return useCallback(
    (outline, opts) => generateSceneContent(outline, aiCall, opts),
    [aiCall],
  );
}

```

This hook pattern enables **declarative scene generation** within React components while maintaining proper dependency management.

## Key Source Files Reference

| File Path | Role in `generateSceneContent` Architecture |
|-----------|---------------------------------------------|
| `packages/@openmaic/generation/src/scene-generator.ts` | Core implementation with dispatch logic and helper functions |
| `packages/@openmaic/generation/src/scene-types.ts` | TypeScript interfaces for `SceneOutline`, `GenerationOptions`, and return types |
| `packages/@openmaic/generation/src/widget-generator.ts` | Contains `generateWidgetContent` for interactive content |
| [`lib/server/scene-generation.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/server/scene-generation.ts) | Production server endpoint consuming the function |
| [`hooks/use-scene-generator.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/hooks/use-scene-generator.ts) | React integration layer |
| `scripts/generation-node-smoke.mjs` | Standalone execution example |

## Summary

- **`generateSceneContent`** centralizes lesson content generation in OpenMAIC through a **type-dispatched architecture**
- The function handles **four major outline types**: interactive widgets, slides, quizzes, and PBL lessons
- **Interactive content** migrates legacy configs and delegates to `generateWidgetContent` for five widget variants
- **Non-interactive content** uses explicit switch-based routing to specialized generators with rich post-processing
- **Media resolution helpers** (`resolveImageIds`, `normalizeGeneratedVideoRefs`, `fixElementDefaults`, `processLatexElements`) ensure LLM output matches production asset realities
- The **unified API surface** supports server routes, React hooks, and CLI scripts without code duplication

## Frequently Asked Questions

### What happens when `generateSceneContent` receives an unknown outline type?

The function returns `null` to signal that no generator exists for the provided type. This explicit null return allows callers to implement fallback behaviors or log diagnostic information about unsupported outline configurations.

### Does `generateSceneContent` handle image generation directly?

No. The function receives pre-existing images through `imageMapping` and `generatedMediaMapping` parameters. It resolves placeholder IDs (like `img_1`) to actual URLs but delegates actual image creation to upstream AI call implementations or external services.

### How does backward compatibility work for interactive content?

Legacy outlines containing `interactiveConfig` objects are automatically migrated via `convertInteractiveConfigToWidget`. Additionally, outlines without an explicit `widgetType` default to `'simulation'`, ensuring older content continues to function without manual updates.

### Can `generateSceneContent` operate without a logger?

Yes. The function uses **nullish coalescing** (`options.logger ?? noopGenerationLogger`) to substitute a no-op logger when none is provided. This eliminates mandatory logging configuration while preserving structured logging capabilities for production deployments.