# How to Add Support for New Animation Engines in video-use: Beyond the Big Four

> Extend video-use with new animation engines. Implement a slot-based architecture and leverage the ffmpeg pipeline to add custom animations.

- Repository: [Browser Use/video-use](https://github.com/browser-use/video-use)
- Tags: how-to-guide
- Published: 2026-07-04

---

**To add support for new animation engines in the browser-use/video-use repository, you implement a slot-based architecture that treats each animation as an independent overlay rendered externally, requiring updates to documentation, installation guides, and slot scaffolding while leveraging the existing ffmpeg compositing pipeline in [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py).**

The browser-use/video-use repository currently orchestrates four animation engines—**HyperFrames**, **Remotion**, **Manim**, and **PIL**—through a decentralized slot system. To add support for new animation engines beyond these four, you must follow the repository's **lazy-install, slot-first** philosophy, which cleanly separates animation generation logic from final video compositing.

## Understanding the Slot-Based Architecture

The core of video-use does not implement animation logic directly. Instead, it manages **slot creation**, **lazy installation** of dependencies, and **final compositing** of overlay videos. Each animation lives in `edit/animations/slot_<id>/` as an independent unit that renders to `render.mp4` or `render.webm` for alpha channels.

According to [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) at line 19, the system "Generates animation overlays via HyperFrames, Remotion, Manim, or PIL". The [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) file consumes these overlays through an ffmpeg filter graph without engine-specific processing, reading paths directly from the EDL's `overlays` field as documented in the top comment (lines 1-8).

## Step-by-Step Guide to Adding a New Engine

### 1. Declare the Engine in SKILL.md

Update [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) at line 19 to include your new engine in the supported engines table. This allows users to select it when authoring a slot by specifying the engine name in the slot configuration.

```markdown
| Core | MyEngine | Scene rendering, animation engine |

```

Add this entry near the existing core entries in the table.

### 2. Document Lazy Installation Requirements

As noted in [`install.md`](https://github.com/browser-use/video-use/blob/main/install.md) at line 160, "HyperFrames, Remotion, and Manim are optional animation engines". Add a bullet describing your new engine's runtime requirements, dependencies, and the exact installation command(s) needed for first-use setup. This maintains the repository's **lazy-install** model where engines are only installed when first needed.

### 3. Create Slot Scaffolding

When users create animation slots, the repository expects the directory `edit/animations/slot_<id>/` to contain everything the engine needs, including source files, configuration, and a build script. Mimic the HyperFrames scaffolding pattern documented at [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) line 210.

Your scaffolding helper should:

- Detect the selected engine from the slot's [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) entry (e.g., `engine: myengine`)
- Run the engine's CLI to render a video into `render.mp4` (or `render.webm` for alpha)

```bash

# In the edit/animations directory

mkdir -p slot_3 && cd slot_3

# Initialise a MyEngine project (replace with the real init command)

myengine init . --template blank

# Edit the source files, then render

myengine render . -o render.mp4

# Write the overlay entry back to the EDL (handled by the sub-agent)

```

### 4. Hook Into the Rendering Pipeline

The [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) compositing step requires **no engine-specific code**. It builds the final video by concatenating graded clips and overlaying animation videos in a single ffmpeg filter graph. Simply ensure your slot's output path is written to the EDL under the `overlays` field so [`render.py`](https://github.com/browser-use/video-use/blob/main/render.py) will pick it up automatically.

```json
{
  "overlays": [
    {
      "file": "edit/animations/slot_3/render.mp4",
      "start_in_output": 12.3,
      "duration": 5.0
    }
  ]
}

```

When [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) runs, it automatically inserts this overlay into the ffmpeg filter chain.

### 5. Update Documentation and Validation

Update [`README.md`](https://github.com/browser-use/video-use/blob/main/README.md) at line 19 to list the new engine alongside the existing four in the feature list. Describe any special considerations such as required Node.js versions, Python dependencies, or GPU support requirements.

### 6. Optional: Provide a Wrapper Library

If your engine exposes a complex API (e.g., a Python library rather than a CLI), add a tiny wrapper module such as [`helpers/myengine.py`](https://github.com/browser-use/video-use/blob/main/helpers/myengine.py). This wrapper converts the slot's JSON description into the engine's native format and invokes the library. The sub-agent (the LLM-driven process that runs the engine) imports this wrapper, but [`render.py`](https://github.com/browser-use/video-use/blob/main/render.py) does not need to call it directly.

## Summary

- **Update [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md)** at line 19 to declare the new engine in the supported engines table.
- **Document installation** in [`install.md`](https://github.com/browser-use/video-use/blob/main/install.md) at line 160 following the lazy-install pattern.
- **Create scaffolding** that generates `render.mp4` in `edit/animations/slot_<id>/` following the pattern at [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) line 210.
- **Integrate with the pipeline** by writing overlay paths to the EDL; [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) handles compositing automatically.
- **Update [`README.md`](https://github.com/browser-use/video-use/blob/main/README.md)** at line 19 to reflect the new capability in the feature list.
- **Optional:** Add a wrapper module in `helpers/` for complex library-based engines.

## Frequently Asked Questions

### Do I need to modify [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) to add a new animation engine?

No. The [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) file uses a generic ffmpeg filter graph that consumes overlay paths from the EDL. As long as your engine outputs a standard video file (MP4 or WebM) and the path is correctly entered in the EDL's `overlays` field, the rendering pipeline will composite it automatically without engine-specific modifications.

### What file format should the animation engine output?

The engine must output either `render.mp4` for standard video or `render.webm` for videos requiring alpha transparency. These files must be placed in the slot directory (`edit/animations/slot_<id>/`). The compositing pipeline in [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) reads these files and layers them over the base video using ffmpeg.

### How does video-use handle dependencies for new engines?

The repository follows a **lazy installation** model. According to [`install.md`](https://github.com/browser-use/video-use/blob/main/install.md) line 160, animation engines are optional and installed only when first needed. When adding support for new animation engines, you must document the installation commands in [`install.md`](https://github.com/browser-use/video-use/blob/main/install.md) so the sub-agent can execute them on first use of that engine type.

### Can I use a Python library instead of a CLI tool for my animation engine?

Yes. If your engine requires a Python API rather than a command-line interface, create a wrapper module in `helpers/` (e.g., [`helpers/myengine.py`](https://github.com/browser-use/video-use/blob/main/helpers/myengine.py)). This module translates the slot's JSON configuration into the library's required format. The sub-agent imports and calls this wrapper during the rendering phase, while [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) continues to work with the resulting video file.