How @openmaic/generation Orchestrates AI Model Calls: Inside the OpenMAIC Engine

The @openmaic/generation package orchestrates AI model calls through a seven-stage pipeline that constructs templated prompts, invokes a configurable AICallFn, repairs malformed JSON responses, and normalizes outputs into strongly typed educational content objects.

The @openmaic/generation package serves as the core orchestration layer within the THU-MAIC/OpenMAIC repository, transforming high-level instructional requirements into concrete slides, quizzes, and interactive widgets. It abstracts the complexity of large language model interactions by encapsulating the entire LLM workflow—from prompt construction to domain-specific post-processing—into a modular, TypeScript-based engine.

The Seven-Stage Orchestration Pipeline

The package implements a robust workflow that handles every aspect of AI content generation. Each stage is designed to be interchangeable, allowing developers to swap underlying models or extend support for new content types without disrupting the core architecture.

Stage 1: Prompt Template Loading and Variable Interpolation

The orchestration begins with prompt construction in packages/@openmaic/generation/src/prompts/loader.js. The buildPrompt function loads templated system and user prompts, then interpolates required variables—such as title, outline, image lists, and language directives—into the template structure. These templates are registered via PROMPT_IDS in packages/@openmaic/generation/src/prompts/index.ts, enabling type-safe selection of the appropriate prompt for each generation task.

Stage 2: Model Invocation via the AICallFn Interface

The package decouples itself from specific LLM providers through the AICallFn interface defined in packages/@openmaic/generation/src/pipeline-types.ts. This generic function signature accepts (system, user, images?) and returns Promise<string>, allowing the host application to inject any compatible AI client. The orchestrator invokes this function via aiCall directly within generation functions like generateSlideContent and generateSceneOutlinesFromRequirements in packages/@openmaic/generation/src/scene-generator.ts and packages/@openmaic/generation/src/outline-generator.ts, optionally passing vision images for multimodal models.

Stage 3: Response Parsing and JSON Repair

Raw LLM responses often contain malformed JSON or extraneous markdown formatting. The parseJsonResponse utility in packages/@openmaic/generation/src/json-repair.ts handles robust parsing with automatic error recovery. For interactive widget generation, the extractHtml function (also in scene-generator.ts) isolates HTML content from model responses, ensuring downstream processors receive clean, structured data regardless of the LLM's output format.

Stage 4: Content Normalization and Asset Resolution

After parsing, the package executes domain-specific post-processing to normalize generated content. Key functions in packages/@openmaic/generation/src/scene-generator.ts include:

• fixElementDefaults – Fills missing required fields with sensible defaults.
• processLatexElements – Renders mathematical expressions via KaTeX.
• resolveImageIds – Maps abstract image IDs to concrete URLs using the provided imageMapping.
• normalizeGeneratedVideoRefs – Standardizes video reference formats across different scene types.

Stage 5: Resilient Execution with Retry Logic

Transient failures such as rate limits or network timeouts are handled by callWithRetry in packages/@openmaic/generation/src/generation-retry.ts. This utility detects retry-able errors using a regex-based classifier and implements exponential backoff. When retries are exhausted, the orchestrator wraps failures into typed error objects like SceneContentFailure or PBLGenerationError, optionally invoking fallback planners for project-based learning (PBL) content.

Stage 6: Dynamic Scene Type Dispatch

The generateSceneContent function acts as a central router (lines 27–78 in scene-generator.ts), dispatching to specialized generators based on the scene type:

• generateSlideContent for presentation slides.
• generateQuizContent for assessment items.
• generateWidgetContent for ultra-mode interactive elements (simulation, diagram, code, game, visualization3d, procedural-skill).
• generatePBLSceneContent for project-based learning modules.

Stage 7: Type-Safe Output Generation

Each generator returns a strongly typed object defined in packages/@openmaic/generation/src/scene-types.ts. These include GeneratedSlideContent, GeneratedQuizContent, GeneratedInteractiveContent, and GeneratedPBLContent, ensuring that downstream rendering engines receive predictably structured data with full TypeScript intellisense support.

Practical Implementation Examples

Generating Slide Content from an Outline

import { generateSceneContent } from '@openmaic/generation';
import { myAICall } from './my-ai-client';

// `outline` is a SceneOutline produced by the outline generator
const slide = await generateSceneContent(outline, myAICall, {
  imageMapping,         // map of image IDs → URLs
  visionEnabled: true,   // pass vision images if the model supports them
  logger: consoleLogger,
});

This call internally builds the prompt using PROMPT_IDS.SLIDE_CONTENT, invokes myAICall, parses the JSON response, fixes defaults, resolves image IDs, renders LaTeX, and returns a GeneratedSlideContent object.

Implementing Resilient AI Calls with Retry Logic

import { callWithRetry } from '@openmaic/generation/src/generation-retry';

const response = await callWithRetry(
  async () => myAICall(systemPrompt, userPrompt, visionImages),
  { maxAttempts: 3, logger: consoleLogger }
);

The callWithRetry wrapper detects transient errors via the regex defined in generation-retry.ts and automatically retries the operation up to the configured limit before throwing a terminal error.

Creating Interactive Ultra-Mode Widgets

import { generateWidgetContent } from '@openmaic/generation';

const widget = await generateWidgetContent(outline, myAICall, undefined, {
  allowProceduralSkill: true,
  logger: consoleLogger,
});

console.log(widget.html);          // HTML returned by the LLM
console.log(widget.widgetConfig); // Optional JSON config embedded in the HTML

This function selects the appropriate PROMPT_IDS (e.g., DIAGRAM_CONTENT), constructs the prompt, calls the model, extracts the HTML using extractHtml, and returns the widget payload with configuration metadata.

Core Files and Architecture

The orchestration logic is distributed across specialized modules for maintainability:

• packages/@openmaic/generation/src/scene-generator.ts – Central dispatcher containing generateSceneContent and scene-specific generators (lines 27–78), plus post-processing utilities like fixElementDefaults and extractHtml.
• packages/@openmaic/generation/src/outline-generator.ts – Generates scene outlines from user requirements, serving as the entry point for the orchestration pipeline.
• packages/@openmaic/generation/src/prompts/loader.js – Implements buildPrompt for template loading and variable interpolation.
• packages/@openmaic/generation/src/prompts/index.ts – Defines PROMPT_IDS and exports prompt-building utilities.
• packages/@openmaic/generation/src/json-repair.ts – Contains parseJsonResponse for tolerant JSON parsing.
• packages/@openmaic/generation/src/generation-retry.ts – Implements callWithRetry with regex-based error classification for transient failures.
• packages/@openmaic/generation/src/scene-types.ts – Type definitions for all generated content variants.
• packages/@openmaic/generation/src/pipeline-types.ts – Defines the AICallFn contract for model agnosticism.
• packages/@openmaic/generation/src/prompt-formatters.ts – Helper functions for assembling context (teacher personas, agent configurations).
• packages/@openmaic/generation/src/interactive-post-processor.ts – Sanitizes and validates HTML for interactive widgets.

Summary

• The @openmaic/generation package provides a complete orchestration framework for AI-driven educational content, handling everything from prompt construction to error recovery.
• It uses a generic AICallFn interface to remain agnostic of specific LLM providers while supporting multimodal inputs including vision images.
• Robust parsing utilities like parseJsonResponse and extractHtml ensure reliable handling of imperfect LLM outputs.
• The architecture supports six distinct scene types through a dynamic dispatch system in scene-generator.ts, delivering strongly typed results for each category.
• Automatic retry logic with exponential backoff mitigates transient failures, improving reliability in production environments.

Frequently Asked Questions

How does @openmaic/generation handle malformed JSON responses from AI models?

The package uses the parseJsonResponse utility in packages/@openmaic/generation/src/json-repair.ts to tolerate malformed JSON by attempting repairs before parsing. For HTML extraction in widget generation, the extractHtml function isolates markup from surrounding text, ensuring downstream processors receive valid content even when the LLM includes explanatory markdown or code fences.

What types of educational content can the orchestration engine generate?

The system supports four primary content categories defined in scene-types.ts: slides (GeneratedSlideContent), quizzes (GeneratedQuizContent), interactive widgets (GeneratedInteractiveContent including simulations, diagrams, and games), and project-based learning scenes (GeneratedPBLContent). The generateSceneContent router automatically dispatches to the appropriate generator based on the scene type specified in the outline.

Can developers swap the underlying LLM provider without modifying the core logic?

Yes. The package accepts an AICallFn implementation from the host application, defined in pipeline-types.ts as (system: string, user: string, images?: string[]) => Promise<string>. This dependency injection pattern allows seamless swapping between OpenAI, Anthropic, Gemini, or self-hosted models without changing the orchestration code in scene-generator.ts.

How does the retry mechanism determine which errors are transient?

The callWithRetry function in packages/@openmaic/generation/src/generation-retry.ts uses a regex-based classifier to identify retry-able conditions such as rate limits, network timeouts, and temporary service unavailability. When detected, it implements exponential backoff with configurable maxAttempts, wrapping terminal failures into typed errors like SceneContentFailure for upstream handling.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}