How Animation Slots Integrate with HyperFrames, Remotion, Manim, and PIL in video-use

video-use treats every animation as an isolated slot inside edit/animations/, where each engine generates a self-contained render that the EDL references as an overlay in helpers/render.py.

The browser-use/video-use repository orchestrates complex video edits by delegating visual overlays to modular animation slots. Regardless of whether you author with HyperFrames, Remotion, Manim, or PIL, each slot lives in its own sub-directory under edit/animations/ and produces a render file that the core pipeline composites automatically. This design keeps animation dependencies lazy, engine-agnostic, and parallelizable.

What Is an Animation Slot?

An animation slot is a dedicated sub-directory inside the session's edit/animations/ folder, conventionally named slot_1/, slot_2/, and so on. According to SKILL.md, each slot contains everything needed to build a short video clip that later gets overlaid on the final edit. The only contract is that the slot must produce a rendered video file—usually render.mp4 or render.webm—that the Edit Decision List (EDL) can reference.

HyperFrames Slot Integration

Setting Up a HyperFrames Slot

Inside a slot directory, initialize a blank HyperFrames project and author your animation with HTML, CSS, and GSAP:

mkdir -p edit/animations/slot_hf
cd edit/animations/slot_hf

npx --yes hyperframes init . --example blank --non-interactive --skip-skills

# Edit index.html / style.css / script.js to create the animation

# ...

npx --yes hyperframes render . -o render.mp4

As noted in SKILL.md, you can also pass --format webm when you need an alpha channel. The entire HyperFrames toolchain is installed lazily inside the slot and never pollutes the repository root.

HyperFrames and the EDL

Once render.mp4 exists, the slot path is stored in the EDL's overlays array. For example:

{
  "file": "edit/animations/slot_1/render.mp4",
  "start_in_output": 0.0,
  "duration": 5.0
}

helpers/render.py reads this overlays list, adds each file as an extra -i input, and shifts it with setpts=PTS-STARTPTS+T/TB so that frame 0 lands exactly at the overlay window start. This timing logic is defined in Hard Rule 4 of SKILL.md and implemented in helpers/render.py between lines 21 and 27.

Remotion Slot Integration

Setting Up a Remotion Slot

Remotion slots follow the same self-contained pattern. Scaffold a local React composition inside a new slot:

mkdir -p edit/animations/slot_remotion
cd edit/animations/slot_remotion

npx create-video@latest

# Edit src/Video.tsx to define the composition

# ...

npx remotion render src/Video.tsx 10 render.mp4

SKILL.md documents this workflow as the standard way to author Remotion-based animations without interleaving node_modules with the main project.

Remotion and the EDL

The rendered render.mp4 path is placed in the EDL overlays entry exactly like HyperFrames. Because helpers/render.py is engine-agnostic, any Remotion clip is treated as a regular video file. The same ffmpeg overlay pipeline consumes it with no special handling required.

Manim Slot Integration

Setting Up a Manim Slot

video-use ships a ready-made skill under skills/manim-video/. Create a slot, add a scene script, and render:

mkdir -p edit/animations/slot_manim
cd edit/animations/slot_manim

cat > scene.py <<'PY'
from manim import *
class Intro(Scene):
    def construct(self):
        txt = Text("Hello, video-use!").scale(2)
        self.play(Write(txt))
        self.wait(2)
PY

manim -pql scene.py Intro
cp media/videos/scene/Intro.mp4 render.mp4

The skills/manim-video/SKILL.md file provides additional guidance on scene planning and rendering parameters.

Manim and the EDL

After copying the output to render.mp4, the slot is referenced in the EDL overlays array just like HyperFrames and Remotion. helpers/render.py composites the Manim overlay using the generic ffmpeg filter chain at lines 21 through 27.

PIL Slot Integration

Setting Up a PIL Slot

For quick cards or typewriter effects, a Python script can draw frames with Pillow and pipe them directly to ffmpeg:

from pathlib import Path
from PIL import Image, ImageDraw, ImageFont
import subprocess

out = Path("render.mp4")
frame = Image.new("RGB", (1280, 720), (10, 10, 10))
draw = ImageDraw.Draw(frame)
font = ImageFont.truetype("/System/Library/Fonts/Menlo.ttc", 80)
draw.text((640, 360), "01", font=font, fill="white", anchor="mm")
frame.save("frame0.png")

subprocess.run([
    "ffmpeg", "-y", "-f", "image2pipe", "-i", "pipe:", "-t", "5",
    "-c:v", "libx264", "-pix_fmt", "yuv420p", str(out)
], input=frame.tobytes())

The script writes the output into the slot directory as render.mp4, satisfying the same slot contract as the other engines.

PIL and the EDL

Because PIL produces a standard MP4, the resulting render.mp4 is added to the EDL overlays field without any special flags. The engine-agnostic overlay logic in helpers/render.py handles the rest.

How helpers/render.py Composites Overlays

The final assembly stage reads the EDL's overlays array and builds an ffmpeg filter graph. For each overlay entry:

• It adds the slot video as an extra -i input.
• It applies setpts=PTS-STARTPTS+T/TB to shift the clip so its first frame aligns with start_in_output.
• It composites the overlay before subtitles, per Hard Rule 1 in SKILL.md.

These constraints are enforced in helpers/render.py between lines 21 and 45. The pipeline remains identical whether the source is HyperFrames, Remotion, Manim, or PIL.

Key Architectural Rules

The video-use animation system is governed by a few hard rules that keep the pipeline deterministic:

• Lazy, per-slot installation. HyperFrames, Remotion, and Manim are not installed globally. They are pulled into a slot only when that slot first needs them, as documented in install.md lines 53 through 60.
• Parallel sub-agents. When an EDL contains multiple overlays, a separate sub-agent is spawned for each slot (Hard Rule 10 in SKILL.md line 31). All agents run concurrently, so wall time is bounded by the slowest animation.
• Overlay-before-subtitle ordering. Overlays are always applied before subtitles burn into the final output (Hard Rule 1, SKILL.md lines 22 through 23).
• EDL-driven workflow. The final video moves through three deterministic stages: per-segment extraction, lossless concatenation, overlay compositing, optional subtitle burn, and optional loudness normalization.

Summary

• video-use manages animations as self-contained slots under edit/animations/.
• Each slot produces a render.mp4 (or .webm) that the EDL references via the overlays array.
• HyperFrames slots scaffold HTML/CSS/GSAP compositions with hyperframes init and hyperframes render.
• Remotion slots contain local React projects rendered via npx remotion render.
• Manim slots leverage the skills/manim-video/ skill and render Python scenes.
• PIL slots generate frames programmatically and pipe them to ffmpeg.
• helpers/render.py composites every overlay engine-agnostically using setpts=PTS-STARTPTS+T/TB.

Frequently Asked Questions

Why doesn't video-use install animation engines globally?

Engines are installed lazily inside each slot to avoid polluting the repository root and to keep startup dependencies minimal. This per-slot isolation is described in install.md lines 53 through 60 and allows different slots to use different engine versions without conflict.

How does overlay timing stay synchronized with the main video?

helpers/render.py applies the ffmpeg expression setpts=PTS-STARTPTS+T/TB to each overlay input. This shifts the overlay timeline so that its frame 0 aligns exactly with the start_in_output timestamp declared in the EDL, as mandated by Hard Rule 4 in SKILL.md.

Can multiple animation slots render at the same time?

Yes. Hard Rule 10 in SKILL.md line 31 states that the skill spawns a separate sub-agent for each slot, and all agents run in parallel. The total wall time is therefore bounded by the slowest slot rather than the sum of all slots.

Is PIL handled differently from HyperFrames or Remotion during compositing?

No. Because every slot ultimately produces a standard video file, helpers/render.py treats all engines identically. The ffmpeg overlay pipeline at lines 21 through 27 consumes HyperFrames, Remotion, Manim, and PIL renders through the same filter graph with no special-casing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}