# How the OpenMAIC Preview Rendering Pipeline Uses Chromium for Scene Previews

> Discover how OpenMAIC uses Chromium and Puppeteer in its preview rendering pipeline to capture PNG screenshots of scene previews for THU-MAIC OpenMAIC.

- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: internals
- Published: 2026-09-13

---

**The OpenMAIC preview rendering pipeline launches a headless Chromium instance via Puppeteer to render self-contained HTML pages for individual scenes, waits for all assets to settle, and captures a PNG screenshot for the caller.**

The preview rendering pipeline in the [THU-MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC) repository generates deterministic image previews of interactive educational content by orchestrating a complete browser lifecycle inside the `render-service` container. Implemented in [`render-service/src/preview-renderer.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/render-service/src/preview-renderer.ts), the system creates isolated HTML documents, executes them in a sandboxed Chromium environment with strict resource budgets, and extracts rasterized outputs without external network dependencies.

## Pipeline Architecture Overview

The core implementation resides in the `ChromiumPreviewRenderer` class, which manages the entire lifecycle from markup generation to browser cleanup. The pipeline supports three distinct scene types—**slide**, **interactive**, and **cover**—each requiring specialized HTML assembly but sharing the same browser execution strategy.

Key design constraints include:
- **Single-use browser instances**: Each preview request launches a fresh Chromium process to ensure isolation
- **AbortSignal integration**: All async operations respect cancellation tokens to prevent hanging processes
- **Asset stabilization**: The system waits for fonts, images, and DOM mutations to settle before capture
- **Network isolation**: The containerized renderer operates without egress to external sites

## HTML Document Generation

### Dynamic Markup Assembly

The pipeline begins by constructing a self-contained HTML document via `buildPreviewHtml`. This function delegates to scene-specific markup generators:

- **`slidePreviewMarkup`**: Creates a `<div>` container that hosts the React-based slide client
- **`interactivePreviewMarkup`**: Wraps embedded HTML inside a sandboxed `<iframe>` with a storage shim
- **`coverPreviewMarkup`**: Renders static cover pages for quizzes and PBL scenarios

All variants are wrapped by `htmlDocument` to produce a standards-compliant HTML5 document complete with necessary metadata and styling constraints.

### Client Bundling with Esbuild

For slide scenes, the renderer bundles a lightweight React client using **esbuild** via `buildSlideClientBundle`. This process:
1. Compiles the `SlideCanvas` component and its dependencies
2. Caches the resulting bundle in the module-level variable `slideClientBundle` for reuse across requests
3. Injects the bundle into the page during the mounting phase

## Browser Lifecycle Management

### Executable Resolution and Launch

The `ChromiumPreviewRenderer.render` method resolves the Chromium binary path from environment variables `PRODUCER_HEADLESS_SHELL_PATH` or `PUPPETEER_EXECUTABLE_PATH`. It then invokes `browserLauncher.launch` with minimal security flags:

```typescript
--no-sandbox
--disable-dev-shm-usage

```

The launch promise is wrapped by `launchWithAbort`, enabling immediate cancellation if the caller's `AbortSignal` triggers before the browser initializes.

### Viewport Configuration

After launching, the pipeline creates a new page via `browser.newPage()` and applies the requested viewport dimensions:

- **width** and **height**: Define the screenshot resolution
- **deviceScaleFactor**: Controls pixel density for high-DPI outputs

This ensures the rendered output matches the UI dimensions specified in the preview request.

### Content Injection Strategies

The HTML document generated in the first phase is injected via `page.setContent`, which waits for the `domcontentloaded` event. For slide scenes, the renderer subsequently executes `mountSlideClient`, which:
1. Assigns scene data to `window.__OPENMAIC_PREVIEW_PROPS__`
2. Appends the bundled script to the document head
3. Polls until `window.__OPENMAIC_PREVIEW_MOUNTED__` is set by the client, indicating successful React hydration

## Stabilization and Capture

### Scene Verification

Before capturing, the pipeline executes a verification check via `page.evaluate` to ensure the `<body>` element's `data-scene-id` attribute matches the requested scene identifier. This guards against race conditions where the wrong content might render in reused page contexts.

### Asset Settling Logic

The pipeline implements sophisticated waiting mechanisms to ensure visual completeness:

- **Interactive scenes**: `waitForInteractiveFrame` waits for the iframe to load, then delegates to `waitForDocumentAssets`
- **Slide and cover scenes**: Calls `waitForDocumentAssets` directly

The `waitForDocumentAssets` function employs a `MutationObserver` to detect when DOM mutations cease, combined with explicit waits for font loading and image decoding. This prevents screenshots from capturing partially loaded states or placeholder content.

### Screenshot Extraction

Once assets stabilize, the pipeline invokes:

```typescript
page.screenshot({ type: 'png', optimizeForSpeed: true })

```

This produces a PNG buffer that is wrapped in a `Uint8Array` and returned to the caller. The `optimizeForSpeed` flag prioritizes generation velocity over compression ratio, appropriate for preview generation workflows.

## Error Handling and Resource Cleanup

### AbortSignal Integration

All long-running operations are wrapped by `raceWithAbort`, which races the rendering promise against the caller's `AbortSignal`. If cancellation occurs:
1. The browser launch or rendering promise rejects immediately
2. The underlying browser process is force-killed
3. A `PreviewTimeoutError` is thrown to the caller

### Bounded Browser Termination

Regardless of success or failure, the pipeline ensures cleanup via `closeBrowserBounded`. This function imposes a strict timeout on `browser.close()` to prevent zombie processes if Chromium hangs during shutdown. The bounded timeout guarantees resource reclamation even under adverse conditions.

## Implementation Examples

### Service Integration

The following pattern from [`render-service/src/main.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/render-service/src/main.ts) demonstrates typical usage:

```typescript
import { ChromiumPreviewRenderer } from './preview-renderer';

const previewRenderer = new ChromiumPreviewRenderer();
const request = {
  scene,
  stage,
  viewport,
  signal: AbortSignal.timeout(20_000), // 20s hard timeout
  deadlineMs: Date.now() + 20_000,
};

const pngBytes = await previewRenderer.render(request);
// pngBytes is a Uint8Array containing the PNG preview image

```

### Testing the Renderer

Unit tests validate the full pipeline integration:

```typescript
import { ChromiumPreviewRenderer } from './preview-renderer';

test('renders a slide scene to PNG', async () => {
  const renderer = new ChromiumPreviewRenderer();
  const png = await renderer.render({
    scene: sampleSlideScene,
    stage: sampleStage,
    viewport: { width: 1920, height: 1080, deviceScaleFactor: 2 },
    signal: new AbortController().signal,
    deadlineMs: Date.now() + 10_000,
  });
  
  expect(png).toBeInstanceOf(Uint8Array);
  expect(png.length).toBeGreaterThan(0);
});

```

## Summary

- The `ChromiumPreviewRenderer` class in [`render-service/src/preview-renderer.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/render-service/src/preview-renderer.ts) orchestrates the entire preview pipeline
- **Three scene types** (slide, interactive, cover) share a unified HTML generation and capture strategy
- **Puppeteer** launches isolated Chromium instances with `--no-sandbox` and `--disable-dev-shm-usage` flags for containerized environments
- **Asset stabilization** via `waitForDocumentAssets` and `MutationObserver` ensures screenshots capture fully rendered frames
- **AbortSignal integration** via `raceWithAbort` and `launchWithAbort` prevents resource exhaustion from hanging requests
- **Environment variables** `PRODUCER_HEADLESS_SHELL_PATH` and `PUPPETEER_EXECUTABLE_PATH` configure the browser binary location

## Frequently Asked Questions

### What environment variables configure the Chromium executable path?

The renderer checks `PRODUCER_HEADLESS_SHELL_PATH` first, falling back to `PUPPETEER_EXECUTABLE_PATH` if the former is unset. These variables must point to a Chromium or Chrome binary compatible with the Puppeteer version specified in the project dependencies.

### How does the pipeline prevent hanging processes during rendering?

All async operations respect an `AbortSignal` passed by the caller. The `raceWithAbort` wrapper cancels pending operations if the signal triggers, while `closeBrowserBounded` enforces a hard timeout on browser shutdown. This dual-layer protection ensures that even if Chromium becomes unresponsive, the container process reclaims resources within a bounded timeframe.

### What is the difference between interactive and slide scene rendering?

**Interactive scenes** inject content into a sandboxed `<iframe>` using `interactivePreviewMarkup` and wait for frame-specific assets via `waitForInteractiveFrame`. **Slide scenes** compile a React client bundle with esbuild, mount it into a dedicated `<div>`, and verify hydration via `window.__OPENMAIC_PREVIEW_MOUNTED__`. Both paths ultimately use the same screenshot capture mechanism once assets stabilize.

### How does the system verify the correct scene is being previewed?

Before capturing the screenshot, the pipeline executes `page.evaluate` to check that `document.body.getAttribute('data-scene-id')` matches the requested scene identifier. This verification step ensures that staged content mutations or client-side navigation errors do not result in mismatched preview outputs.