How the OpenMAIC Preview Rendering Pipeline Uses Chromium for Scene Previews

The OpenMAIC preview rendering pipeline launches a headless Chromium instance via Puppeteer to render self-contained HTML pages for individual scenes, waits for all assets to settle, and captures a PNG screenshot for the caller.

The preview rendering pipeline in the THU-MAIC/OpenMAIC repository generates deterministic image previews of interactive educational content by orchestrating a complete browser lifecycle inside the render-service container. Implemented in render-service/src/preview-renderer.ts, the system creates isolated HTML documents, executes them in a sandboxed Chromium environment with strict resource budgets, and extracts rasterized outputs without external network dependencies.

Pipeline Architecture Overview

The core implementation resides in the ChromiumPreviewRenderer class, which manages the entire lifecycle from markup generation to browser cleanup. The pipeline supports three distinct scene types—slide, interactive, and cover—each requiring specialized HTML assembly but sharing the same browser execution strategy.

Key design constraints include:

• Single-use browser instances: Each preview request launches a fresh Chromium process to ensure isolation
• AbortSignal integration: All async operations respect cancellation tokens to prevent hanging processes
• Asset stabilization: The system waits for fonts, images, and DOM mutations to settle before capture
• Network isolation: The containerized renderer operates without egress to external sites

HTML Document Generation

Dynamic Markup Assembly

The pipeline begins by constructing a self-contained HTML document via buildPreviewHtml. This function delegates to scene-specific markup generators:

• slidePreviewMarkup: Creates a <div> container that hosts the React-based slide client
• interactivePreviewMarkup: Wraps embedded HTML inside a sandboxed <iframe> with a storage shim
• coverPreviewMarkup: Renders static cover pages for quizzes and PBL scenarios

All variants are wrapped by htmlDocument to produce a standards-compliant HTML5 document complete with necessary metadata and styling constraints.

Client Bundling with Esbuild

For slide scenes, the renderer bundles a lightweight React client using esbuild via buildSlideClientBundle. This process:

1. Compiles the SlideCanvas component and its dependencies
2. Caches the resulting bundle in the module-level variable slideClientBundle for reuse across requests
3. Injects the bundle into the page during the mounting phase

Browser Lifecycle Management

Executable Resolution and Launch

The ChromiumPreviewRenderer.render method resolves the Chromium binary path from environment variables PRODUCER_HEADLESS_SHELL_PATH or PUPPETEER_EXECUTABLE_PATH. It then invokes browserLauncher.launch with minimal security flags:

--no-sandbox
--disable-dev-shm-usage

The launch promise is wrapped by launchWithAbort, enabling immediate cancellation if the caller's AbortSignal triggers before the browser initializes.

Viewport Configuration

After launching, the pipeline creates a new page via browser.newPage() and applies the requested viewport dimensions:

• width and height: Define the screenshot resolution
• deviceScaleFactor: Controls pixel density for high-DPI outputs

This ensures the rendered output matches the UI dimensions specified in the preview request.

Content Injection Strategies

The HTML document generated in the first phase is injected via page.setContent, which waits for the domcontentloaded event. For slide scenes, the renderer subsequently executes mountSlideClient, which:

1. Assigns scene data to window.__OPENMAIC_PREVIEW_PROPS__
2. Appends the bundled script to the document head
3. Polls until window.__OPENMAIC_PREVIEW_MOUNTED__ is set by the client, indicating successful React hydration

Stabilization and Capture

Scene Verification

Before capturing, the pipeline executes a verification check via page.evaluate to ensure the <body> element's data-scene-id attribute matches the requested scene identifier. This guards against race conditions where the wrong content might render in reused page contexts.

Asset Settling Logic

The pipeline implements sophisticated waiting mechanisms to ensure visual completeness:

• Interactive scenes: waitForInteractiveFrame waits for the iframe to load, then delegates to waitForDocumentAssets
• Slide and cover scenes: Calls waitForDocumentAssets directly

The waitForDocumentAssets function employs a MutationObserver to detect when DOM mutations cease, combined with explicit waits for font loading and image decoding. This prevents screenshots from capturing partially loaded states or placeholder content.

Screenshot Extraction

Once assets stabilize, the pipeline invokes:

page.screenshot({ type: 'png', optimizeForSpeed: true })

This produces a PNG buffer that is wrapped in a Uint8Array and returned to the caller. The optimizeForSpeed flag prioritizes generation velocity over compression ratio, appropriate for preview generation workflows.

Error Handling and Resource Cleanup

AbortSignal Integration

All long-running operations are wrapped by raceWithAbort, which races the rendering promise against the caller's AbortSignal. If cancellation occurs:

1. The browser launch or rendering promise rejects immediately
2. The underlying browser process is force-killed
3. A PreviewTimeoutError is thrown to the caller

Bounded Browser Termination

Regardless of success or failure, the pipeline ensures cleanup via closeBrowserBounded. This function imposes a strict timeout on browser.close() to prevent zombie processes if Chromium hangs during shutdown. The bounded timeout guarantees resource reclamation even under adverse conditions.

Implementation Examples

Service Integration

The following pattern from render-service/src/main.ts demonstrates typical usage:

import { ChromiumPreviewRenderer } from './preview-renderer';

const previewRenderer = new ChromiumPreviewRenderer();
const request = {
  scene,
  stage,
  viewport,
  signal: AbortSignal.timeout(20_000), // 20s hard timeout
  deadlineMs: Date.now() + 20_000,
};

const pngBytes = await previewRenderer.render(request);
// pngBytes is a Uint8Array containing the PNG preview image

Testing the Renderer

Unit tests validate the full pipeline integration:

import { ChromiumPreviewRenderer } from './preview-renderer';

test('renders a slide scene to PNG', async () => {
  const renderer = new ChromiumPreviewRenderer();
  const png = await renderer.render({
    scene: sampleSlideScene,
    stage: sampleStage,
    viewport: { width: 1920, height: 1080, deviceScaleFactor: 2 },
    signal: new AbortController().signal,
    deadlineMs: Date.now() + 10_000,
  });
  
  expect(png).toBeInstanceOf(Uint8Array);
  expect(png.length).toBeGreaterThan(0);
});

Summary

• The ChromiumPreviewRenderer class in render-service/src/preview-renderer.ts orchestrates the entire preview pipeline
• Three scene types (slide, interactive, cover) share a unified HTML generation and capture strategy
• Puppeteer launches isolated Chromium instances with --no-sandbox and --disable-dev-shm-usage flags for containerized environments
• Asset stabilization via waitForDocumentAssets and MutationObserver ensures screenshots capture fully rendered frames
• AbortSignal integration via raceWithAbort and launchWithAbort prevents resource exhaustion from hanging requests
• Environment variables PRODUCER_HEADLESS_SHELL_PATH and PUPPETEER_EXECUTABLE_PATH configure the browser binary location

Frequently Asked Questions

What environment variables configure the Chromium executable path?

The renderer checks PRODUCER_HEADLESS_SHELL_PATH first, falling back to PUPPETEER_EXECUTABLE_PATH if the former is unset. These variables must point to a Chromium or Chrome binary compatible with the Puppeteer version specified in the project dependencies.

How does the pipeline prevent hanging processes during rendering?

All async operations respect an AbortSignal passed by the caller. The raceWithAbort wrapper cancels pending operations if the signal triggers, while closeBrowserBounded enforces a hard timeout on browser shutdown. This dual-layer protection ensures that even if Chromium becomes unresponsive, the container process reclaims resources within a bounded timeframe.

What is the difference between interactive and slide scene rendering?

Interactive scenes inject content into a sandboxed <iframe> using interactivePreviewMarkup and wait for frame-specific assets via waitForInteractiveFrame. Slide scenes compile a React client bundle with esbuild, mount it into a dedicated <div>, and verify hydration via window.__OPENMAIC_PREVIEW_MOUNTED__. Both paths ultimately use the same screenshot capture mechanism once assets stabilize.

How does the system verify the correct scene is being previewed?

Before capturing the screenshot, the pipeline executes page.evaluate to check that document.body.getAttribute('data-scene-id') matches the requested scene identifier. This verification step ensures that staged content mutations or client-side navigation errors do not result in mismatched preview outputs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}