Video-Use Animation Technologies: HyperFrames, Remotion, Manim, and PIL Explained

Video-use supports four interchangeable animation engines—HyperFrames, Remotion, Manim, and PIL + ffmpeg—that generate overlay animations via isolated slots in edit/animations/slot_<id>/ directories.

The browser-use/video-use repository is a conversation-driven video editing framework that integrates multiple animation technologies through a slot-based architecture. Each animation engine produces a render.mp4 file that the central pipeline consumes, allowing you to mix HTML-based motion graphics, React compositions, mathematical visualizations, and programmatic frame sequences in a single project.

Overview of Animation Technologies

Video-use implements a slot-based architecture where every animation lives in its own folder under edit/animations/slot_<id>/. This isolation enables the Agent tool to spawn parallel sub-agents for each slot, satisfying the parallel processing requirements defined in the production rules.

All four engines are optional and lazy-installed according to install.md. They are pulled into the project only when a specific slot requires them, keeping the base installation lightweight. The final ffmpeg concat chain applies subtitles after every overlay, ensuring that graphics never obscure text.

HyperFrames for Browser-Native Compositions

HyperFrames generates HTML/CSS/GSAP compositions for UI mock-ups, kinetic typography, and browser-native animations. It requires Node.js 22+ and renders via the HyperFrames CLI.

To create a HyperFrames slot, scaffold the project inside the slot directory and render to the standard output format:

mkdir -p edit/animations/slot_01
cd edit/animations/slot_01
npx --yes hyperframes init . --example blank --non-interactive --skip-skills

# Edit index.html and src files as needed

npx --yes hyperframes render . -o render.mp4

The resulting render.mp4 is referenced in edl.json as an overlay entry. HyperFrames supports alpha channels via WebM output when transparency is required.

Remotion for React-Based Video

Remotion enables React-based video compositions using standard React components, CSS, and reusable primitives. It is ideal for developers familiar with React who want to build dynamic video content programmatically.

Initialize a Remotion project inside a slot directory:

mkdir -p edit/animations/slot_02
cd edit/animations/slot_02
npx create-video@latest

# Edit src/Composition.jsx to build your animation

npx remotion render src/Composition.jsx 30

This produces render.mp4 in the slot directory, which helpers/render.py consumes during the final composite. Remotion slots follow the same overlay ordering rules as other engines.

Manim for Mathematical Animations

Manim provides programmatic mathematical and diagrammatic animations for equations, graphs, and state machines. The repository ships a dedicated skill at skills/manim-video/ with specific authoring instructions.

Navigate to the skill directory and author your scene:

cd skills/manim-video

# Write my_scene.py following the skill's SKILL.md references

manim -ql my_scene.py MyScene -o ../../edit/animations/slot_03/render.mp4

The output path places the rendered video directly into the slot structure for EDL integration. Manim is particularly effective for educational content and technical explanations requiring precise mathematical visualization.

PIL and FFmpeg for Frame-by-Frame Overlays

PIL + ffmpeg generates simple frame-by-frame overlays such as counters, typewriter text, progress bars, and progressive draws. This approach uses the Pillow library to create PNG sequences, then stitches them with ffmpeg.

The helper scripts helpers/render.py and helpers/grade.py contain the plumbing for this workflow. Create a Python script to generate your sequence:

from pathlib import Path
from PIL import Image, ImageDraw, ImageFont

out_dir = Path("edit/animations/slot_04")
out_dir.mkdir(parents=True, exist_ok=True)

font = ImageFont.truetype("/System/Library/Fonts/Menlo.ttc", 48)
for i in range(60):  # 2-second 30 fps sequence

    img = Image.new("RGBA", (1280, 720), (0, 0, 0, 0))
    draw = ImageDraw.Draw(img)
    draw.text((640, 360), f"{i//30 + 1}", font=font, anchor="mm", fill="white")
    img.save(out_dir / f"{i:04d}.png")

# Stitch with ffmpeg (called by the helper render script)

# ffmpeg -r 30 -i slot_04/%04d.png -c:v libx264 -pix_fmt yuv420p render.mp4

This method offers maximum control for simple graphics without requiring external Node.js or Python dependencies beyond Pillow and ffmpeg.

Slot Architecture and EDL Integration

The slot-based architecture enforces strict isolation between animation sources. Each slot contains:

• Engine-specific source files
• A render.mp4 (or .webm) output
• An overlay.json entry consumed by the main pipeline

When a video contains multiple animation slots, video-use launches one sub-agent per slot in parallel. The slowest slot determines overall wall-clock time, optimizing for concurrent execution.

The edl.json description is engine-agnostic, referencing only the file path and timing. As implemented in helpers/render.py, the rendering step does not care which engine produced the file, enabling seamless mixing of technologies:

{
  "type": "overlay",
  "file": "edit/animations/slot_01/render.mp4",
  "start": 12.34,
  "end": 17.80
}

Summary

• Four interchangeable engines: HyperFrames (HTML/CSS), Remotion (React), Manim (mathematical), and PIL + ffmpeg (frame sequences).
• Lazy installation: Engines are installed on-demand per install.md, keeping the base project lightweight.
• Slot isolation: Each animation lives in edit/animations/slot_<id>/ with its own render.mp4 output.
• Parallel processing: Multiple slots render concurrently via sub-agents, with the slowest slot determining total time.
• EDL agnostic: The edl.json format references only file paths and timing, allowing mixed-engine projects.
• Subtitle safety: The ffmpeg concat chain applies subtitles after all overlays, preventing text obstruction.

Frequently Asked Questions

How do I choose between the four animation engines?

HyperFrames suits browser-native UI demonstrations and GSAP animations. Remotion is optimal for React developers building component-based video content. Manim excels at mathematical equations and technical diagrams. PIL + ffmpeg handles simple programmatic graphics like counters and progress bars without heavy dependencies. According to SKILL.md, you can select engines per-slot based on the specific visual requirements of each scene.

Can I mix different animation technologies in a single video?

Yes. Because each slot is completely isolated in edit/animations/slot_<id>/, you can combine engines within one project. For example, use a PIL title card in slot 1, a HyperFrames UI demo in slot 2, and a Manim graph in slot 3. The edl.json references only the output render.mp4 files, making the final composite engine-agnostic.

Are the animation engines installed by default?

No. As noted in install.md, animation engines are lazy-installed. They are pulled into the project only when you initialize a slot that requires them. This keeps the initial installation lightweight while allowing access to heavy dependencies like Node.js 22+ for HyperFrames or the Manim Python environment when needed.

How does video-use handle subtitles with animation overlays?

The rendering pipeline in helpers/render.py applies an ffmpeg concat chain that processes all overlay slots first, then applies subtitles after every overlay. This hard rule ensures that animation graphics never obscure subtitle text, maintaining accessibility and readability regardless of which animation technology generated the overlay.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}