How OpenScreen Implements Crop Regions for Video Recordings: A Technical Deep Dive

OpenScreen implements crop regions using normalized coordinates (0-1 range) stored in the editor state, applying them to both the live Pixi.js preview via sprite positioning and masking, and to the export pipeline through canvas clipping to ensure what you see is what you get.

OpenScreen handles video cropping through a dual-path architecture that guarantees visual consistency between the editor preview and the final exported file. The implementation relies on a CropRegion interface using normalized floating-point values to maintain resolution independence across different source videos. By processing these coordinates separately in the live renderer and the export encoder, the application ensures that hidden portions of the video recording remain excluded from both the interactive preview and the output media.

Crop Region Data Structure and State Management

The foundation of OpenScreen's cropping system lives in the core type definitions, where coordinates are stored as ratios rather than absolute pixels.

Normalized Coordinate System

In src/components/video-editor/types.ts, the application defines a CropRegion interface that uses normalized values between 0 and 1:

// src/components/video-editor/types.ts
export interface CropRegion {
  x: number;      // normalized left offset (0-1)
  y: number;      // normalized top offset (0-1)
  width: number;  // normalized width (0-1)
  height: number; // normalized height (0-1)
}

export const DEFAULT_CROP_REGION: CropRegion = { 
  x: 0, 
  y: 0, 
  width: 1, 
  height: 1 
};

This normalization ensures the crop region scales correctly regardless of the source video resolution. The UI components mutate this object via pushState({ cropRegion: r }), triggering re-renders in both the preview and export pipelines.

Live Preview Rendering with Pixi.js

The live preview applies the crop region through three distinct steps in src/components/video-editor/videoPlayback/layoutUtils.ts: dimension calculation, sprite positioning, and visual masking.

Calculating Cropped Dimensions

The layoutVideoContent function first converts normalized coordinates to pixel values based on the source video dimensions:

// src/components/video-editor/videoPlayback/layoutUtils.ts
const crop = cropRegion || { x: 0, y: 0, width: 1, height: 1 };
const croppedVideoWidth = videoWidth * crop.width;
const croppedVideoHeight = videoHeight * crop.height;
const cropStartX = crop.x * videoWidth;
const cropStartY = crop.y * videoHeight;

This conversion happens within the layout utility (lines 70-84), establishing the effective display size of the visible video area.

Sprite Positioning and Offset Logic

To center the cropped region within the on-screen display rectangle, the system calculates precise sprite offsets:

const offsetX = screenRect.x + (screenRect.width - croppedVideoWidth * scale) / 2;
const offsetY = screenRect.y + (screenRect.height - croppedVideoHeight * scale) / 2;
const spriteX = offsetX - crop.x * fullVideoDisplayWidth;
const spriteY = offsetY - crop.y * fullVideoDisplayHeight;
videoSprite.position.set(spriteX, spriteY);

This arithmetic ensures the video sprite shifts so that only the designated crop area aligns with the visible screen rectangle, effectively hiding the portions outside the crop region.

Masking Implementation

A Pixi.js Graphics mask provides the final visual clipping layer:

// src/components/video-editor/videoPlayback/layoutUtils.ts
maskGraphics.clear();
maskGraphics.roundRect(
  screenRect.x,
  screenRect.y,
  screenRect.width,
  screenRect.height,
  compositeLayout.screenCover ? 0 : borderRadius
);
maskGraphics.fill({ color: 0xffffff });

The mask clips all rendering to the screen rectangle boundaries, guaranteeing that any video content outside the crop region remains invisible to the user.

Export Pipeline Frame Rendering

During export, src/lib/exporter/frameRenderer.ts processes the same normalized crop region to clip frames before encoding.

Canvas Clipping During Encoding

The frame renderer applies the crop region via the Canvas API's drawImage method, using the source coordinate parameters to extract only the desired portion:

// src/lib/exporter/frameRenderer.ts
const { cropRegion, borderRadius = 0, padding = 0 } = this.config;
const cropStartX = cropRegion.x;
const cropStartY = cropRegion.y;
const cropEndX = cropRegion.x + cropRegion.width;
const cropEndY = cropRegion.y + cropRegion.height;

// Drawing the cropped frame
ctx.drawImage(
  sourceVideo,
  videoWidth * cropStartX,                // source X
  videoHeight * cropStartY,               // source Y
  videoWidth * (cropEndX - cropStartX),   // source width
  videoHeight * (cropEndY - cropStartY),  // source height
  destX, destY, destWidth, destHeight     // destination on canvas
);

This approach (lines 417-425) ensures the exported MP4 or GIF contains exactly the pixels visible in the editor preview, maintaining frame-perfect consistency.

User Interface and State Updates

The crop region interface connects to React state through specific components in the video editor.

Settings Panel Integration

The SettingsPanel component renders a crop control modal that updates the editor state:

// src/components/video-editor/SettingsPanel.tsx
<CropControl
  videoElement={videoElement}
  cropRegion={cropRegion}
  onCropChange={(newRegion) => pushState({ cropRegion: newRegion })}
/>

When users adjust the crop rectangle in CropControl, the onCropChange callback persists the normalized region to the editor state, immediately propagating changes to both the Pixi.js preview container in VideoPlayback.tsx and the export configuration object.

Summary

• Normalized storage: OpenScreen stores crop regions as x, y, width, and height values between 0 and 1 in src/components/video-editor/types.ts, ensuring resolution independence.
• Dual application: The same crop region feeds both the live Pixi.js renderer (layoutVideoContent in layoutUtils.ts) and the export canvas renderer (frameRenderer.ts).
• Visual masking: The live preview uses a Graphics mask to clip the video sprite to the screen rectangle while offsetting the sprite position to center the cropped content.
• Canvas clipping: During export, drawImage source coordinates extract the exact pixel region defined by the normalized crop values.
• State synchronization: The SettingsPanel and CropControl components mutate the crop region through pushState, keeping the preview and export pipelines synchronized.

Frequently Asked Questions

What coordinate system does OpenScreen use for crop regions?

OpenScreen uses a normalized coordinate system where all values (x, y, width, height) are floating-point numbers between 0 and 1, representing ratios of the source video dimensions. This implementation in src/components/video-editor/types.ts allows crop regions to remain valid regardless of video resolution changes.

How does the crop region affect the exported video file?

During export, src/lib/exporter/frameRenderer.ts converts the normalized crop region into pixel coordinates and passes them to the Canvas API's drawImage method as source clipping parameters. This ensures the encoded MP4 or GIF contains only the pixels within the defined crop boundary, permanently excluding hidden portions from the final output.

Why does OpenScreen apply the crop in both the preview and export stages?

OpenScreen maintains separate rendering paths for performance and fidelity. The Pixi.js preview uses sprite positioning and masking for real-time 60fps interaction, while the export pipeline uses precise canvas clipping to ensure frame-perfect output. Both systems consume the same normalized CropRegion state to guarantee WYSIWYG consistency.

What happens if no crop region is defined in the editor state?

The application defaults to DEFAULT_CROP_REGION { x: 0, y: 0, width: 1, height: 1 }, which represents the full video frame. Both layoutVideoContent in layoutUtils.ts and frameRenderer.ts check for the presence of a crop region and fall back to these default values, ensuring the entire video remains visible when no cropping is active.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}