# How Annotation Regions in OpenScreen Support Text, Images, and Figure Overlays with Styling

> Discover how OpenScreen annotation regions unify text, image, and figure overlays with precise styling support. Explore canvas export and live UI rendering capabilities.

- Repository: [Sid/openscreen](https://github.com/siddharthvaddem/openscreen)
- Tags: how-to-guide
- Published: 2026-04-03

---

**OpenScreen implements a unified `AnnotationRegion` type system that stores position, timing, and styling data for text, image, and figure overlays, with specialized renderers in [`annotationRenderer.ts`](https://github.com/siddharthvaddem/openscreen/blob/main/annotationRenderer.ts) and [`AnnotationOverlay.tsx`](https://github.com/siddharthvaddem/openscreen/blob/main/AnnotationOverlay.tsx) handling both canvas export and live UI interactions.**

OpenScreen is an open-source video editing platform that enables precise overlay management through its annotation region architecture. Each overlay is stored as an `AnnotationRegion` object defined in [`src/components/video-editor/types.ts`](https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/types.ts), specifying content type, visual styling, temporal boundaries, and spatial coordinates. This system supports three distinct overlay types—**text**, **image**, and **figure**—with dedicated rendering pipelines for real-time editing and final video export.

## The AnnotationRegion Type System

The foundation of OpenScreen's overlay capability lies in the `AnnotationRegion` interface. This type encapsulates all metadata required to position and style overlays on the video canvas using percentage-based coordinates.

Each region specifies its **position** (`x` and `y` as percentages of the video dimensions), **size** (`width` and `height` as percentages), **z-index** for stacking order, and optional **start/end timestamps** (`startMs` and `endMs`) to control temporal visibility.

The `type` field accepts one of three `AnnotationType` values, each with corresponding styling data:

- **`text`**: Renders styled text using `AnnotationTextStyle`, which includes `color`, `backgroundColor`, `fontSize`, `fontFamily`, `fontWeight`, `fontStyle`, `textDecoration`, and `textAlign`.
- **`image`**: Displays an image data-URL (`data:image/*`) using object-contain scaling within the region bounds, ignoring text styling properties.
- **`figure`**: Renders directional arrows (e.g., "up-right") using `FigureData`, which specifies `arrowDirection`, `color`, and `strokeWidth`.

## Rendering Architecture: Three-Layer Pipeline

OpenScreen processes annotation regions through three coordinated layers that ensure consistent rendering across editing and export workflows.

### Canvas Export Layer

During video export, [`src/lib/exporter/annotationRenderer.ts`](https://github.com/siddharthvaddem/openscreen/blob/main/src/lib/exporter/annotationRenderer.ts) manages the rendering pipeline. The module iterates over `annotationRegions`, filters entries by the current timestamp, sorts them by `zIndex`, and dispatches to type-specific render helpers:

- **`renderText`**: Draws styled text on a `<canvas>` element using the region's `style` fields.
- **`renderImage`**: Loads the image data-URL and preserves aspect ratio within the region bounds.
- **`renderArrow`**: Retrieves SVG path data from the internal `ARROW_PATHS` table and strokes the arrow using the region's `figureData` properties for color and stroke width.

This canvas-based approach ensures overlays appear in the final exported video with exact styling fidelity.

### Live Editing Layer

While editing, [`src/components/video-editor/AnnotationOverlay.tsx`](https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/AnnotationOverlay.tsx) provides the interactive interface. The component wraps each region in a draggable, resizable `<Rnd>` (react-rnd) container that:

- Converts percentage-based coordinates to pixel values for the current viewport.
- Conditionally renders the appropriate inner element based on `annotation.type`: a `<span>` for text, `<img>` for images, or an arrow component imported from [`src/components/video-editor/ArrowSvgs.tsx`](https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/ArrowSvgs.tsx).
- Applies CSS styling directly to the rendered element, mapping `annotation.style` properties to inline styles (e.g., `color`, `fontSize`, `backgroundColor`).

This layer enables direct manipulation of position and size while preserving the styling defined in the region's type system.

### Playback Integration Layer

The [`src/components/video-editor/VideoPlayback.tsx`](https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/VideoPlayback.tsx) component orchestrates overlay visibility during playback. It receives `annotationRegions` via props, filters them against the current playback time to determine active overlays, sorts by `zIndex`, and renders an `AnnotationOverlay` instance for each visible region.

This layer also handles click-through cycling for overlapping annotations, temporarily boosting the selected annotation's z-index to ensure it remains accessible during editing.

## Implementation Examples

### Defining Annotation Regions

To create overlays, define objects conforming to the `AnnotationRegion` interface:

```typescript
import {
  type AnnotationRegion,
  type AnnotationTextStyle,
  type FigureData,
} from "@/components/video-editor/types";

const textStyle: AnnotationTextStyle = {
  color: "#ffffff",
  backgroundColor: "rgba(0,0,0,0.5)",
  fontSize: 36,
  fontFamily: "Inter",
  fontWeight: "bold",
  fontStyle: "italic",
  textDecoration: "underline",
  textAlign: "center",
};

export const myAnnotations: AnnotationRegion[] = [
  // Text overlay
  {
    id: "anno-1",
    startMs: 2000,
    endMs: 8000,
    type: "text",
    content: "Hello, Open Screen!",
    position: { x: 50, y: 20 },
    size: { width: 40, height: 15 },
    style: textStyle,
    zIndex: 10,
  },

  // Image overlay
  {
    id: "anno-2",
    startMs: 5000,
    endMs: 12000,
    type: "image",
    content: "data:image/png;base64,iVBORw0KG…",
    position: { x: 20, y: 60 },
    size: { width: 30, height: 30 },
    style: textStyle,
    zIndex: 20,
  },

  // Figure (arrow) overlay
  {
    id: "anno-3",
    startMs: 0,
    endMs: 15000,
    type: "figure",
    content: "",
    position: { x: 70, y: 70 },
    size: { width: 20, height: 20 },
    style: textStyle,
    figureData: {
      arrowDirection: "up-right",
      color: "#34B27B",
      strokeWidth: 4,
    } as FigureData,
    zIndex: 30,
  },
];

```

The `AnnotationRegion` interface is defined in [`src/components/video-editor/types.ts`](https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/types.ts).

### Integrating with Video Playback

Pass the annotation array to the `VideoPlayback` component:

```tsx
import VideoPlayback from "@/components/video-editor/VideoPlayback";

<VideoPlayback
  videoPath="/videos/sample.mp4"
  webcamLayoutPreset="picture-in-picture"
  zoomRegions={[]}
  selectedZoomId={null}
  onSelectZoom={() => {}}
  onZoomFocusChange={() => {}}
  isPlaying={true}
  aspectRatio="16:9"
  annotationRegions={myAnnotations}
  selectedAnnotationId={null}
  onSelectAnnotation={(id) => console.log("Selected", id)}
  onAnnotationPositionChange={(id, pos) => {
    // Update region position in state
  }}
  onAnnotationSizeChange={(id, size) => {
    // Update region size in state
  }}
/>

```

The `annotationRegions` prop is consumed in [`src/components/video-editor/VideoPlayback.tsx`](https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/VideoPlayback.tsx) to render active overlays.

### Applying Styles in Live Overlays

Inside [`AnnotationOverlay.tsx`](https://github.com/siddharthvaddem/openscreen/blob/main/AnnotationOverlay.tsx), text styling is applied via inline CSS:

```tsx
<span
  style={{
    color: annotation.style.color,
    backgroundColor: annotation.style.backgroundColor,
    fontSize: `${annotation.style.fontSize}px`,
    fontFamily: annotation.style.fontFamily,
    fontWeight: annotation.style.fontWeight,
    fontStyle: annotation.style.fontStyle,
    textDecoration: annotation.style.textDecoration,
    textAlign: annotation.style.textAlign,
    lineHeight: "1.4",
    padding: "0.1em 0.2em",
    borderRadius: "4px",
  }}
>
  {annotation.content}
</span>

```

For figure rendering during export, [`annotationRenderer.ts`](https://github.com/siddharthvaddem/openscreen/blob/main/annotationRenderer.ts) invokes:

```typescript
renderArrow(
  ctx,
  annotation.figureData.arrowDirection,
  annotation.figureData.color,
  annotation.figureData.strokeWidth,
  x,
  y,
  width,
  height,
  scaleFactor,
);

```

## Summary

- **AnnotationRegion** objects in [`src/components/video-editor/types.ts`](https://github.com/siddharthvaddem/openscreen/blob/main/src/components/video-editor/types.ts) provide a type-safe foundation for text, image, and figure overlays with full styling control through `AnnotationTextStyle` and `FigureData`.
- The three-layer rendering pipeline—canvas export ([`annotationRenderer.ts`](https://github.com/siddharthvaddem/openscreen/blob/main/annotationRenderer.ts)), live editing ([`AnnotationOverlay.tsx`](https://github.com/siddharthvaddem/openscreen/blob/main/AnnotationOverlay.tsx)), and playback integration ([`VideoPlayback.tsx`](https://github.com/siddharthvaddem/openscreen/blob/main/VideoPlayback.tsx))—ensures consistent behavior across export and editing modes.
- **Temporal control** via `startMs` and `endMs` enables time-bound overlays that appear only during specific video segments.
- **Z-index management** allows precise stacking control, with automatic sorting by `zIndex` and selection boosting in the playback layer to handle overlapping regions.
- **Figure overlays** utilize pre-computed SVG path tables for directional arrows with customizable stroke width and color.

## Frequently Asked Questions

### How does OpenScreen handle overlapping annotation regions?

OpenScreen resolves overlapping annotations through **z-index sorting**. In both [`annotationRenderer.ts`](https://github.com/siddharthvaddem/openscreen/blob/main/annotationRenderer.ts) and [`VideoPlayback.tsx`](https://github.com/siddharthvaddem/openscreen/blob/main/VideoPlayback.tsx), regions are sorted by their `zIndex` property before rendering. During live editing, [`VideoPlayback.tsx`](https://github.com/siddharthvaddem/openscreen/blob/main/VideoPlayback.tsx) implements click-through cycling that temporarily increases the selected annotation's z-index, ensuring users can select and edit regions even when they overlap completely.

### What image formats are supported for image overlays?

OpenScreen accepts any image format that can be encoded as a **data-URL** (`data:image/*`). The `content` field of an `image` type `AnnotationRegion` expects a base64-encoded data string (e.g., `data:image/png;base64,iVBORw0KG…`). The renderer in [`annotationRenderer.ts`](https://github.com/siddharthvaddem/openscreen/blob/main/annotationRenderer.ts) uses standard Canvas API image loading, which supports PNG, JPEG, WebP, and other browser-compatible formats. The image is drawn using object-contain logic to preserve aspect ratio within the defined region bounds.

### Can text annotations use custom fonts in the exported video?

Yes, provided the font is loaded in the browser context before export. The `fontFamily` property in `AnnotationTextStyle` accepts any CSS font family name. During canvas export in [`annotationRenderer.ts`](https://github.com/siddharthvaddem/openscreen/blob/main/annotationRenderer.ts), the renderer applies the specified `fontFamily` to the canvas context. For consistent rendering between the live editor and final export, ensure custom fonts are preloaded using CSS `@font-face` or similar mechanisms before initiating the export process.

### How are figure arrows positioned and oriented within their regions?

Figure arrows are positioned to fill the entire annotation region bounds. The `figureData.arrowDirection` property (e.g., "up-right", "down-left") selects a pre-computed SVG path from the `ARROW_PATHS` table in the renderer. The arrow is scaled to fit the region's `width` and `height` while maintaining stroke consistency according to `strokeWidth`. The `color` property defines the stroke color applied during canvas rendering in the `renderArrow` function.