# How to Add Animation Overlays in video-use Without Breaking the Render Pipeline

> Learn how to add animation overlays in video-use without breaking the render pipeline. Discover the correct method using standalone MP4 or WebM files and precise EDL timing for seamless integration.

- Repository: [Browser Use/video-use](https://github.com/browser-use/video-use)
- Tags: how-to-guide
- Published: 2026-07-04

---

**The correct way to add animation overlays in video-use is to generate standalone MP4 or WebM files in isolated animation slots under `edit/animations/slot_<id>/`, then reference them in the `overlays` array of your [`edl.json`](https://github.com/browser-use/video-use/blob/main/edl.json) with precise `start_in_output` and `duration` values, allowing the `build_final_composite` function in [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) to handle PTS synchronization automatically.**

The `video-use` repository (browser-use/video-use) processes video through a strict three-stage pipeline that concatenates base footage before compositing any overlays. Because the final compositing step relies on a single `ffmpeg` filter graph, animation overlays must conform to specific format and directory conventions to avoid misalignment or render failures.

## Understanding the video-use Render Pipeline

The render pipeline defined in [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) executes in three distinct phases:

1. **Per-segment extraction & grading** – Each cut is extracted into individual MP4 segments via `extract_segment`.
2. **Loss-less concatenation** – Graded clips are concat-demuxed into a base video file.
3. **Final compositing** – The base video is overlaid with animation clips and finally subtitles via `build_final_composite` (lines 96-108).

According to the video-use source code, the `build_final_composite` function constructs an `ffmpeg` filter graph where each overlay video is **PTS-shifted** using `setpts=PTS-STARTPTS+<t>/TB` to align its first frame with the `start_in_output` timestamp defined in your EDL. Subtitles are always appended last (lines 38-44), ensuring they appear above all animations.

## Animation Slot Isolation and Directory Structure

The pipeline requires strict **slot isolation** for external animation engines. As documented in [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) and [`install.md`](https://github.com/browser-use/video-use/blob/main/install.md), you must create an animation slot under `edit/animations/slot_<id>/` rather than placing files at the repository root.

- **HyperFrames, Remotion, and Manim** are treated as optional engines installed **lazily** on first use.
- Each slot operates as an isolated workspace where the engine outputs a `render.mp4` or `render.webm` file.
- The [`edl.json`](https://github.com/browser-use/video-use/blob/main/edl.json) references these outputs using relative paths from the EDL location.

## Supported Animation Engines

### HyperFrames Overlays

HyperFrames generates transparent WebM files suitable for overlay graphics. The engine runs via `npx` without global installation:

```bash
mkdir -p edit/animations/slot_1
cd edit/animations/slot_1

# Scaffold and render (lazy install)

npx --yes hyperframes init . --example blank --non-interactive --skip-skills
npx --yes hyperframes render . -o render.webm --format webm

```

### Remotion Overlays

Remotion uses React components to generate video. Scaffold a local project within a slot:

```bash
mkdir -p edit/animations/slot_2 && cd edit/animations/slot_2

# Scaffold locally

npx create-video@latest .

# Render exact duration (e.g., 3 seconds, no audio)

npx remotion render src/Video.jsx 0 3 --output render.mp4

```

Ensure your component renders for exactly the duration specified in your EDL entry.

### Manim Overlays

For mathematical animations, Manim installs via pip on first use:

```bash
mkdir -p edit/animations/slot_3 && cd edit/animations/slot_3
pip install manim

# Create scene file then render

manim example.scene.py Intro -qm -o render.webm --format webm

```

Manim outputs should use WebM format with alpha channels for transparency.

### PIL (Pillow) Static Overlays

For programmatically generated graphics or simple fades, use Pillow to generate frame sequences and encode with `ffmpeg`:

```python
from PIL import Image, ImageDraw, ImageFont
import subprocess
from pathlib import Path

def make_overlay(out_path: Path, duration: float, fps: int = 24):
    frames = int(duration * fps)
    img = Image.new("RGBA", (1920, 1080), (0, 0, 0, 0))
    draw = ImageDraw.Draw(img)
    font = ImageFont.truetype("DejaVuSans-Bold.ttf", 80)
    draw.text((960, 540), "PIL overlay", font=font, fill=(255, 255, 255, 255), anchor="mm")
    
    tmp_dir = out_path.parent / "tmp_frames"
    tmp_dir.mkdir(parents=True, exist_ok=True)
    
    for i in range(frames):
        # Fade-in calculation for first 0.5s

        alpha = int(255 * min(1, (i / (fps * 0.5)))) if i < fps * 0.5 else 255
        frame = img.copy()
        frame.putalpha(alpha)
        frame.save(tmp_dir / f"frame_{i:04d}.png")
    
    subprocess.run([
        "ffmpeg", "-y", "-framerate", str(fps), "-i",
        str(tmp_dir / "frame_%04d.png"),
        "-c:v", "libvpx-vp9", "-pix_fmt", "yuva420p", "-lossless", "1",
        str(out_path)
    ], check=True)
    
    # Cleanup

    for p in tmp_dir.iterdir(): p.unlink()
    tmp_dir.rmdir()

if __name__ == "__main__":
    make_overlay(Path("render.webm"), duration=2.5)

```

Run this script inside an animation slot to produce a transparent WebM ready for compositing.

## EDL Configuration for Overlays

Each overlay requires a JSON entry in the `overlays` list within [`edl.json`](https://github.com/browser-use/video-use/blob/main/edl.json):

```json
{
  "sources": { "take1": "raw/take1.mp4" },
  "ranges": [
    { "source": "take1", "start": 0.0, "end": 6.0 }
  ],
  "overlays": [
    {
      "file": "animations/slot_1/render.webm",
      "start_in_output": 1.5,
      "duration": 4.0
    }
  ]
}

```

**Critical parameters:**
- `file`: Relative path to the animation slot output
- `start_in_output`: Exact timestamp in seconds where frame 0 of the overlay appears
- `duration`: Length of the overlay in seconds (must match the generated video duration)

As implemented in `browser-use/video-use`, the `build_final_composite` function automatically applies the `setpts` filter to synchronize timing. Do **not** pre-offset the video yourself.

## Technical Requirements for Overlay Videos

To prevent pipeline breakage, adhere to these constraints:

- **Resolution**: Match the base video resolution (default 1080p). Mismatched sizes trigger automatic scaling but may degrade quality.
- **Frame rate**: Use 24 fps to match the default base video frame rate.
- **Format**: Use WebM with alpha channel (`yuva420p`) for transparent overlays; MP4 for opaque content.
- **Audio**: Overlays must be silent. The filter graph ignores audio tracks in overlay files.
- **PTS handling**: Let the pipeline manage timestamp shifting via the loop at line 21 of [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py).

## Summary

- **Slot isolation** is mandatory: Create animation outputs under `edit/animations/slot_<id>/` only.
- **Lazy installation** of HyperFrames, Remotion, and Manim occurs on first use; no global installs required.
- **Standalone video files** (MP4/WebM) must match base video resolution and frame rate.
- **EDL entries** require `file`, `start_in_output`, and `duration` parameters; the pipeline handles PTS synchronization automatically.
- **Subtitles render last**, ensuring they appear above all animation overlays.

## Frequently Asked Questions

### Why does my overlay appear at the wrong timestamp?

The `video-use` pipeline uses `setpts=PTS-STARTPTS+<t>/TB` to shift overlay timestamps automatically. If your overlay appears misaligned, verify that the `start_in_output` value in [`edl.json`](https://github.com/browser-use/video-use/blob/main/edl.json) matches the intended output time, and ensure you have not pre-offset the video during generation. The `build_final_composite` function handles synchronization as long as the overlay video starts at 0 seconds relative to its own timeline.

### Can I use transparent backgrounds with animation overlays?

Yes. Use WebM format with alpha channel support (`yuva420p` pixel format) for transparent overlays. HyperFrames generates these by default when using `--format webm`, and the PIL example above uses `libvpx-vp9` with `yuva420p` to preserve transparency. Transparent overlays are composited correctly before the final subtitle layer is applied.

### What happens if my overlay resolution doesn't match the base video?

The `ffmpeg` filter graph in [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) will automatically scale mismatched resolutions, but this may introduce quality degradation or letterboxing. For best results, render overlays at the exact output resolution (default 1920x1080) and 24 fps to avoid any scaling artifacts during the final compositing stage.

### Do I need to install HyperFrames, Remotion, or Manim globally?

No. According to [`install.md`](https://github.com/browser-use/video-use/blob/main/install.md), these engines are **optional** and installed **lazily** per slot. HyperFrames runs via `npx --yes hyperframes`, Remotion via `npx create-video@latest`, and Manim via `pip install manim` executed within the specific animation slot directory. This isolation prevents dependency conflicts with the core video-use pipeline.