# What Types of Interactive UI Elements Does OpenMAIC Support Beyond Standard Slides?
> Discover OpenMAIC's advanced interactive UI elements beyond slides: dynamic HTML scenes, iframe element references, and static offline widgets. Enhance your dashboards now.
- Repository: [MAIC/OpenMAIC](https://github.com/THU-MAIC/OpenMAIC)
- Tags: getting-started
- Published: 2026-09-11
---
**OpenMAIC supports three specialized interactive UI paradigms: dynamic interactive HTML scenes for embedding full widget pages, interactive element references that capture specific clickable DOM nodes inside iframes, and static interactive HTML for frozen offline-capable widgets.**
The OpenMAIC (Machine-Agent Interactive Content) framework distinguishes between static slide objects and dynamic interactive layers that enable user manipulation. While standard slides contain immutable elements like text boxes and images, the repository's [`lib/workbench/element-refs.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/workbench/element-refs.ts) and video-export modules implement APIs for living UI components. This article examines the specific interactive element types available in the `THU-MAIC/OpenMAIC` codebase, including their implementation details and usage patterns.
## Interactive HTML Scenes
Interactive HTML scenes represent self-contained web pages or widgets that render inside the presentation timeline. Unlike static slide objects, these scenes accept HTML content and execute JavaScript within an isolated iframe environment.
In the OpenMAIC source, scenes are defined with the discriminator `type: 'interactive'`, as demonstrated in the video-export test suites. These scenes provide the container for complex UI elements such as buttons, forms, and mini-applications that require DOM interaction.
### Defining an Interactive Scene
The following pattern creates an interactive scene for video compilation:
```typescript
const interactiveScene = {
id: 'interactive:widget',
type: 'interactive',
content: { type: 'interactive', html: '
Widget
' },
};
await compileVideoTimeline({
scenes: [interactiveScene],
// ...other timeline options
});
```
The compiled output places the widget at [`assets/interactive/001-widget.html`](https://github.com/THU-MAIC/OpenMAIC/blob/main/assets/interactive/001-widget.html), which the playback engine loads as a self-contained interactive unit.
## Interactive Element References
While interactive scenes host full HTML pages, **interactive element references** capture specific, clickable DOM nodes within those iframes. The `InteractiveElementRef` type, defined in [`lib/workbench/element-refs.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/workbench/element-refs.ts), records a node's CSS selector, outer HTML, visible text, and localized label.
When a user interacts with the referenced element, the system emits a "click" event that the agent can convert into messages or edit commands. This mechanism allows the LLM to interact with granular UI components rather than entire pages.
### Creating Interactive Element References
Use the `makeInteractiveElementRef` factory function to generate references from user selections:
```typescript
import { makeInteractiveElementRef } from '@/lib/workbench/element-refs';
// Capture a button selected inside an interactive iframe
const picked = {
selector: '#cta-button',
outerHTML: '',
text: 'Click me',
};
const interactiveRef = makeInteractiveElementRef(
'stage-01', // stageId
'scene-interactive-01', // sceneId
picked,
(key) => key, // translation function placeholder
);
// Resulting object structure:
// {
// kind: 'interactive-element',
// stageId: 'stage-01',
// sceneId: 'scene-interactive-01',
// selector: '#cta-button',
// outerHTML: '',
// text: 'Click me',
// label: 'button · Click me',
// }
```
The resulting reference includes `kind: 'interactive-element'`, distinguishing it from standard slide object references according to the implementation in [`tests/workbench/element-refs.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/workbench/element-refs.test.ts).
## Static Interactive HTML
For offline playback and archival purposes, OpenMAIC supports **static interactive HTML**—frozen snapshots of interactive widgets. The `prepareInteractiveHtmlScenes` utility processes dynamic widgets into self-contained ZIP packages that retain their visual state without active server connections.
These static variants include diagnostic attributes like `data-interactive-static-state` to indicate whether the widget is frozen, active, or has fallen back to a degraded state. This functionality is validated in [`tests/video-export/interactive-static-html.browser.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/video-export/interactive-static-html.browser.test.ts), which ensures static widgets render correctly in Chromium environments.
## Standard Slides vs. Interactive UI
Understanding the distinction between static slides and interactive elements is crucial for content architecture:
- **Standard slide elements** include text, images, shapes, and charts that render as static objects without DOM accessibility or event handling.
- **Interactive HTML scenes** provide full iframe encapsulation for custom web applications, supporting JavaScript execution and CSS animations through the `type: 'interactive'` discriminator.
- **Interactive element references** enable granular agent interaction by targeting specific selectors within those iframes, bridging the gap between visual presentation and functional UI manipulation.
- **Static interactive HTML** packages these dynamic experiences into portable, offline-ready formats suitable for video export and distribution.
## Summary
- **Interactive HTML scenes** embed full widget pages via the `type: 'interactive'` property, providing isolated execution environments for custom UI in [`lib/workbench/element-refs.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/workbench/element-refs.ts).
- **Interactive element references** (`InteractiveElementRef`) capture specific clickable DOM nodes using `makeInteractiveElementRef`, enabling precise agent interaction through CSS selectors.
- **Static interactive HTML** freezes widgets for offline use via `prepareInteractiveHtmlScenes`, marked with `data-interactive-static-state` attributes for runtime state management.
- These implementations reside in the `THU-MAIC/OpenMAIC` video-export test suite and element-refs modules, distinguishing OpenMAIC's capabilities from traditional static slide objects.
## Frequently Asked Questions
### What distinguishes an interactive element reference from a standard slide object?
Interactive element references target specific DOM nodes within iframes, storing CSS selectors and outer HTML in the `InteractiveElementRef` structure, whereas standard slide objects represent static, non-interactive content like text boxes or images without underlying DOM accessibility.
### How does OpenMAIC capture user interactions on dynamic UI elements?
When a user clicks an element captured by `makeInteractiveElementRef`, the system extracts the selector and text content from the iframe, creating a reference that emits events the LLM can process as edit commands or messages, as implemented in [`lib/workbench/element-refs.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/lib/workbench/element-refs.ts).
### What is the purpose of static interactive HTML in OpenMAIC?
Static interactive HTML provides frozen, self-contained versions of widgets via `prepareInteractiveHtmlScenes`, allowing presentations to function offline or within exported video timelines while maintaining diagnostic state markers like `data-interactive-static-state`.
### Can interactive HTML scenes execute arbitrary JavaScript?
Yes, interactive scenes defined with `type: 'interactive'` encapsulate full HTML documents within iframes, enabling execution of JavaScript, CSS animations, and complex form handling, validated through the browser-specific tests in [`tests/video-export/interactive-static-html.browser.test.ts`](https://github.com/THU-MAIC/OpenMAIC/blob/main/tests/video-export/interactive-static-html.browser.test.ts).