# How to Extend video-use with Additional Animation Rendering Engines

> Learn how to extend video-use by adding custom animation rendering engines. Follow our guide to register new engines and enhance your video output seamlessly.

- Repository: [Browser Use/video-use](https://github.com/browser-use/video-use)
- Tags: how-to-guide
- Published: 2026-07-03

---

**You can extend video-use by creating a skill directory with a [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) descriptor, a slot-scaffolding script, and registering the engine in the top-level [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md), while the existing [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) pipeline handles the final overlay compositing automatically.**

The `video-use` repository treats every animation as a **slot** that independent rendering engines populate with video files. Because the core pipeline is engine-agnostic, you can integrate any animation tool—from Node.js CLIs to Python packages—without modifying the central rendering logic. This guide walks through the exact file structure, conventions, and code required to extend video-use with new animation rendering engines.

## Understanding the video-use Animation Architecture

### The Slot-Based Workflow

`video-use` manages animations through a strict directory convention. Each animation occupies a unique slot inside `<videos_dir>/edit/animations/slot_<id>/`. According to the source code in [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) (lines 205-212), the agent scaffolds these slots at runtime by executing engine-specific initialization commands.

The workflow follows three distinct phases:

1. **Slot creation** – The agent creates a sub-directory and populates it with starter files specific to the chosen engine (Node.js or Python-based).
2. **Engine invocation** – The slot's build script runs the rendering engine to produce a single video file, typically named `render.mp4` or `render.webm`.
3. **Overlay stitching** – The [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) script receives the rendered file path via the `--overlay` flag, applies a `setpts` timestamp shift, and composites the animation into the final edit.

### The Rendering Pipeline

The central render helper located at [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) implements the `build_final_composite` function (line 575) to handle all overlay processing. This implementation enforces two hard rules automatically:

- **PTS shifting** – Applies `setpts=PTS-STARTPTS+T/TB` to synchronize overlay timing (Rule 4, line 25).
- **Subtitle layering** – Adds subtitles last to ensure they remain visible over animations (Rule 1, line 22).

Because this pipeline only requires a valid video file path, it remains completely decoupled from the specific animation engine used to generate the content.

## Step-by-Step Guide to Adding a New Animation Engine

### Step 1: Create a Skill Sub-Directory

Create a new directory under `skills/<your-engine>/` to house your engine's configuration. This mirrors the existing patterns found in `skills/manim-video/` and `skills/hyperframes/`.

Your skill directory must contain a [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) file describing:

- The engine's purpose and output format
- Required dependencies and installation commands
- Exact CLI commands for slot initialization and rendering

### Step 2: Scaffold the Slot with an Initialization Script

Provide a Bash, Node, or Python script (conventionally named [`init.sh`](https://github.com/browser-use/video-use/blob/main/init.sh)) that creates the slot folder and writes starter project files. The script receives the target slot directory as its first argument (`$1`).

For example, the HyperFrames integration uses `npx --yes hyperframes init` to scaffold browser-based projects. Your script should follow this pattern, preparing all files necessary for the user to edit and render.

### Step 3: Implement the Render Command

Your engine must output exactly one video file (e.g., `render.mp4` or `render.webm`) that the main render helper can consume. The file path is passed to [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) via the `--overlay` parameter during the stitching phase.

Ensure your render command respects the slot directory structure and deposits the final video at the path expected by the agent.

### Step 4: Register the Engine in SKILL.md

Add a reference to your engine in the top-level [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) under the **Animations** section. This registration allows the LLM agent to select your engine during conversation planning, alongside existing options like HyperFrames, Remotion, and Manim.

### Step 5: Document Dependencies in install.md

Update [`install.md`](https://github.com/browser-use/video-use/blob/main/install.md) to mention any additional system dependencies (such as new CLI tools or binary requirements). Following the existing pattern for HyperFrames and Remotion, note that these dependencies will be installed lazily on first use to keep the base installation lightweight.

## Complete Example: Adding an SVG-Animator Engine

Below is a complete implementation adding a fictional `svg-animator` CLI to the video-use pipeline.

First, create the skill directory and descriptor:

```bash
mkdir -p skills/svg-animator/scripts
cat > skills/svg-animator/SKILL.md <<'EOF'
---
name: svg-animator
description: Render SVG-based animations (e.g., animated infographics) into video overlays.
---

# SVG-Animator

* **Dependencies** – `npm i -g svg-animator-cli`
* **Slot scaffold** – `svg-animator init .` creates `src/` with `animation.svg`
* **Render command** – `svg-animator render src/animation.svg -o render.mp4`
EOF

```

Next, create the slot initialization script:

```bash
cat > skills/svg-animator/scripts/init.sh <<'EOF'
#!/usr/bin/env bash
set -e
slot_dir=$1
mkdir -p "$slot_dir/src"
cd "$slot_dir"
svg-animator init .
echo "Slot ready. Edit $slot_dir/src/animation.svg then run:"
echo "  svg-animator render src/animation.svg -o render.mp4"
EOF
chmod +x skills/svg-animator/scripts/init.sh

```

At runtime, the agent executes the workflow as follows:

```bash

# Create and scaffold the slot

slot_dir=$(mktemp -d /tmp/slot_svg_XXXX)
skills/svg-animator/scripts/init.sh "$slot_dir"

# User edits $slot_dir/src/animation.svg, then renders:

svg-animator render "$slot_dir/src/animation.svg" -o "$slot_dir/render.mp4"

# Hand the produced video to the main renderer

python helpers/render.py edl.json -o final.mp4 --overlay "$slot_dir/render.mp4"

```

The [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) script automatically applies the required `setpts` shift and subtitle layering, producing the final composite without engine-specific modifications.

## Key Files and Extension Points

- **[`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) (top-level)** – Defines core animation skills and lists supported engines. Located at the repository root.
- **[`install.md`](https://github.com/browser-use/video-use/blob/main/install.md)** – Documents lazy installation patterns for engine dependencies.
- **[`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py)** – Central render helper that handles overlay stitching and enforces rendering rules (lines 22, 25, and 575).
- **`skills/manim-video/`** – Reference implementation showing a Python-based math animation engine.
- **`skills/hyperframes/`** – Reference for browser-based engines, demonstrating slot-scaffold commands.

## Summary

- **video-use** uses a slot-based architecture where each animation engine populates a directory with a renderable video file.
- To add a new engine, create a `skills/<engine>/` directory containing a [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) descriptor and an initialization script.
- The engine must output a single video file (e.g., `render.mp4`) that [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) composites via the `--overlay` flag.
- Registration requires updating the top-level [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) (for agent selection) and [`install.md`](https://github.com/browser-use/video-use/blob/main/install.md) (for dependency documentation).
- The rendering pipeline in [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) is engine-agnostic, automatically handling timestamp synchronization (`setpts`) and subtitle layering.

## Frequently Asked Questions

### Do I need to modify helpers/render.py to add a new engine?

No. The [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) pipeline is engine-agnostic and consumes any valid video file passed via the `--overlay` flag. You only need to modify this file if you require engine-specific optimizations to the compositing logic, such as custom FFmpeg filters beyond the standard `setpts` shift.

### What video formats are supported for the rendered output?

The overlay system accepts standard video formats including `render.mp4` and `render.webm`. As long as FFmpeg can decode the file, [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) can process it. The engine should deposit the final rendered file in the slot directory with a predictable filename so the agent can reference it correctly.

### How does video-use handle timing synchronization for custom animations?

The [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) script automatically applies a PTS shift using `setpts=PTS-STARTPTS+T/TB` (as implemented at line 25) to align the overlay's timeline with the main video sequence. This ensures that animations render at their intended temporal positions regardless of when the engine generated the file.

### Can I use a Python-based animation library instead of a Node.js CLI?

Yes. The extension architecture supports any rendering engine that produces a video file. Reference the `skills/manim-video/` directory for an example of a Python-based implementation. You would create a Python script instead of a Bash script for slot initialization, but the registration process in [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) and [`install.md`](https://github.com/browser-use/video-use/blob/main/install.md) remains identical.