How OpenScreen Manages Multiple Timeline Region Types: Zoom, Trim, Speed, and Annotations Simultaneously

OpenScreen treats zoom, trim, speed, and annotation regions as independent data arrays stored in a centralized EditorState, merging them during playback through separate lookup functions and compositing them during export via the FrameRenderer class.

The siddharthvaddem/openscreen video editor handles complex timeline manipulations by decoupling region definitions from their runtime application. By storing each region type in its own immutable array within the editor state, the system allows users to stack zoom effects, trimming, speed ramps, and annotations without interference, then resolves these layers deterministically during both real-time preview and final export.

Centralized State Architecture for Timeline Regions

At the core of OpenScreen's region management is a unified state container that isolates each effect type while maintaining their temporal relationships.

The EditorState Interface

In src/hooks/useEditorHistory.ts, the EditorState interface declares separate arrays for every region type, ensuring type-safe storage of overlapping timeline effects:

export interface EditorState {
    zoomRegions: ZoomRegion[];
    trimRegions: TrimRegion[];
    speedRegions: SpeedRegion[];
    annotationRegions: AnnotationRegion[];
    … // other UI state
}

Each array contains plain data objects that define start and end times (in milliseconds) along with type-specific properties like zoom depth, playback speed multiplier, or annotation content.

Immutable History Management

The useEditorHistory hook provides an immutable state reducer through pushState, allowing the UI to modify one region type without mutating others. When a user adds a zoom region from 5–8 seconds and a speed change from 15–18 seconds, the hook appends both states to the history stack independently, enabling precise undo/redo operations per region type.

Playback-Time Region Resolution

During video playback, OpenScreen queries these separate arrays each frame to calculate the current effective state.

Handling Trim and Speed Regions

The createVideoEventHandlers function in src/components/video-editor/videoPlayback/videoEventHandlers.ts applies trimming and speed changes directly to the HTML5 video element:

const activeTrimRegion = findActiveTrimRegion(currentTimeMs);
const activeSpeedRegion = findActiveSpeedRegion(currentTimeMs);
if (activeTrimRegion) video.currentTime = activeTrimRegion.endMs / 1000;
video.playbackRate = activeSpeedRegion ? activeSpeedRegion.speed : 1;

Trim regions work by jumping the playhead past skipped sections, while speed regions modify the playbackRate property dynamically.

Calculating Zoom Transformations

Zoom handling requires interpolation between regions. The findDominantRegion utility in src/components/video-editor/videoPlayback/zoomRegionUtils.ts determines which zoom region (or transition between two) governs the current frame, returning a target scale, focus point, and transition progress. This decoupled approach allows pan transitions between disconnected zoom regions while keeping the calculation separate from playback logic.

Export Pipeline Composition

The FrameRenderer class in src/lib/exporter/frameRenderer.ts orchestrates final output by reusing the same lookup functions used during playback.

FrameRenderer Orchestration

During export initialization, the renderer receives all region arrays:

const renderer = new FrameRenderer({
  width: 1920,
  height: 1080,
  zoomRegions: editorState.zoomRegions,
  trimRegions: editorState.trimRegions,
  speedRegions: editorState.speedRegions,
  annotationRegions: editorState.annotationRegions,
});

For each frame, updateAnimationState(timeMs) calls findDominantRegion to compute the current zoom transform, then applyZoomTransform (from zoomTransform.ts) applies the scale and translation to the canvas context.

Annotation Overlay Rendering

After transforming the video frame, renderAnnotations in src/lib/exporter/annotationRenderer.ts draws active AnnotationRegion objects directly onto the composited canvas. This occurs after zoom application but before final encoding, ensuring text and figures respect the underlying video transforms.

Pre-baked Timeline Effects

Trim and speed modifications are handled upstream in the export pipeline. videoExporter.ts and audioEncoder.ts skip frames falling within trim regions and adjust timestamps according to speed regions before passing data to the FrameRenderer, effectively baking these effects into the input stream rather than the rendering stage.

Working with Multiple Region Types

Developers interact with this system through the React hook API and export utilities.

Adding Regions via useEditorHistory

To programmatically add overlapping timeline effects:

import { useEditorHistory } from "@/hooks/useEditorHistory";

function addSampleRegions() {
  const { pushState } = useEditorHistory();

  // Zoom region with focus point
  pushState({
    zoomRegions: [
      {
        id: "z1",
        startMs: 5000,
        endMs: 8000,
        depth: 3,
        focus: { cx: 0.5, cy: 0.4 },
      },
    ],
  });

  // Trim to skip 12-13 seconds
  pushState({
    trimRegions: [
      { id: "t1", startMs: 12000, endMs: 13000 },
    ],
  });

  // 1.5x speed ramp
  pushState({
    speedRegions: [
      { id: "s1", startMs: 15000, endMs: 18000, speed: 1.5 },
    ],
  });

  // Text annotation overlay
  pushState({
    annotationRegions: [
      {
        id: "a1",
        startMs: 6000,
        endMs: 9000,
        type: "text",
        content: "Hello, OpenScreen!",
        position: { x: 30, y: 70 },
        size: { width: 40, height: 15 },
        style: { color: "#ff6600" },
        zIndex: 10,
      },
    ],
  });
}

Exporting with FrameRenderer

To render all region types to a final video:

import { FrameRenderer } from "@/lib/exporter/frameRenderer";
import { videoExporter } from "@/lib/exporter/videoExporter";

async function exportFullTimeline(editorState: EditorState) {
  const renderer = new FrameRenderer({
    width: 1920,
    height: 1080,
    wallpaper: "/wallpapers/wallpaper1.jpg",
    zoomRegions: editorState.zoomRegions,
    trimRegions: editorState.trimRegions,
    speedRegions: editorState.speedRegions,
    annotationRegions: editorState.annotationRegions,
  });

  await renderer.initialize();
  await videoExporter({
    source: "recorded.webm",
    renderer,
    trimRegions: editorState.trimRegions,
    speedRegions: editorState.speedRegions,
  });

  const finalCanvas = renderer.getCanvas();
  // Process finalCanvas for download
}

Summary

• OpenScreen stores zoom, trim, speed, and annotation regions as separate arrays in the immutable EditorState managed by useEditorHistory.
• During playback, videoEventHandlers.ts consults these arrays independently via findActiveTrimRegion, findActiveSpeedRegion, and findDominantRegion to modify the video element's current time, playback rate, and zoom transform.
• The FrameRenderer class composites final output by applying zoom transformations via applyZoomTransform, then overlaying annotations through renderAnnotations, while trim and speed effects are pre-processed during frame decoding.
• Region data persists across sessions through projectPersistence.ts, which normalizes and reinjects all timeline arrays when loading projects.

Frequently Asked Questions

How does OpenScreen handle overlapping zoom regions on the same timeline?

When multiple zoom regions overlap, findDominantRegion in src/components/video-editor/videoPlayback/zoomRegionUtils.ts calculates which region takes precedence based on start times and transition progress. If regions connect spatially, the utility computes a smooth pan transition between the focus points of the adjacent regions rather than snapping abruptly.

Can trim and speed regions overlap without conflicts?

Yes. Trim regions modify the effective playback position by jumping past skipped sections, while speed regions adjust the playbackRate property. Since videoEventHandlers.ts processes these independently—first checking for active trims to adjust the playhead, then applying speed multipliers—they can overlap without circular dependencies. The exporter handles these sequentially in videoExporter.ts by adjusting timestamps before frame rendering.

Are annotation regions affected by zoom transformations?

Annotations are rendered after zoom transformations are applied to the canvas context. In src/lib/exporter/frameRenderer.ts, the renderAnnotations call occurs following applyZoomTransform, meaning annotation positions are specified in screen coordinates and remain visually anchored to their defined locations regardless of underlying video zoom levels.

How does the undo/redo system track changes across different region types?

The useEditorHistory hook treats the entire EditorState as an immutable snapshot. When pushState is called with a partial update—such as adding a new zoom region while leaving trim data unchanged—the reducer creates a new state object preserving existing arrays. This allows independent undo stacks per region type while maintaining atomic history entries that capture the complete editor state at each step.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}