How video-use Handles Parallel Sub-Agent Animation Spawning for Multiple Overlays

video-use spawns independent sub-agents for each overlay animation in parallel, reducing total wall-time to the duration of the slowest render before compositing everything with FFmpeg.

The browser-use/video-use repository orchestrates complex video editing workflows by treating each overlay animation as an isolated work unit. When an edit-definition-list (EDL) specifies multiple visual overlays, the rendering pipeline leverages parallel sub-agent animation spawning to generate all assets concurrently rather than sequentially. This architecture ensures that adding overlays does not linearly increase render time, allowing complex productions to scale efficiently.

The Parallel Sub-Agent Architecture

One Agent per Overlay

According to SKILL.md, the tool launches a separate sub-agent for each entry in the EDL overlay list using the generic Agent tool supplied by Instagit. As documented at line 31, the workflow mandates: “Parallel sub‑agents for multiple animations. Never sequential. Spawn N at once via the Agent tool; total wall time ≈ slowest one.” Each sub-agent receives a self-contained brief specifying the animation framework and output destination. Because these agents execute in isolated sandboxes, they run simultaneously, and the total wall-time approximates the duration of the slowest individual animation rather than the sum of all renders.

Isolated Brief Structure

To prevent filename collisions and ensure independence, each sub-agent writes its brief to a distinct directory under <edit>/animations/slot_<id>/. As specified in SKILL.md at line 81, this structure allows the Agent tool to spawn N sub-agents at once without resource conflicts. Each brief contains the complete command set, input assets, and target path required for the specific animation framework, ensuring that parallel writes never interfere with one another.

Animation Framework Delegation

The sub-agents are framework-agnostic workers that can invoke HyperFrames, Remotion, Manim, or PIL depending on the overlay requirements. The README.md at line 19 confirms that these are “spawned in parallel sub‑agents, one per animation.” The rendering pipeline delegates the actual pixel generation to these specialized tools, with each sub-agent producing a short video file or image sequence at the path referenced in the EDL entry. This delegation allows different overlays to use different animation engines within the same project.

FFmpeg Composition and Timing Synchronization

Once all parallel sub-agents complete, the main process in helpers/render.py collects the resulting files and composites them onto the base video.

PTS Shifting for Temporal Alignment

The composition engine applies a FFmpeg setpts filter with the expression PTS-STARTPTS+T/TB to shift each overlay's frame 0 to its designated window start time. As noted in SKILL.md at line 25, this overcomes timing drift that could occur if the sub-agent finished rendering at different wall-clock times. The filter ensures that animations align precisely with their scheduled timestamps in the EDL regardless of when the sub-agent actually completed its work.

Layer Ordering and Subtitle Placement

The pipeline iteratively stacks overlays using FFmpeg's overlay filter, applying them in sequence. Crucially, SKILL.md at line 22 mandates that “Subtitles are applied LAST … otherwise overlays hide captions.” This layer ordering prevents opaque animations from obscuring text, ensuring subtitles remain visible in the final composite.

Implementation Examples

Defining Overlays in the EDL

Each overlay entry in the edit-definition-list references the output path where the sub-agent will write its render:

{
  "overlays": [
    {
      "file": "edit/animations/slot_01/render.mp4",
      "start": 12.3,
      "duration": 3.0,
      "style": "bold-overlay"
    },
    {
      "file": "edit/animations/slot_02/render.webm",
      "start": 25.0,
      "duration": 2.5,
      "style": "bold-overlay"
    }
  ]
}

Spawning Parallel Sub-Agents

The top-level script spawns independent agents for each slot, allowing the Instagit framework to execute them in parallel:

from instagit import Agent   # Instagit‑provided tool

def launch_overlay_agent(slot_id, brief_path):
    # `brief_path` is a markdown file with all commands for this overlay

    Agent.run(
        name=f"overlay-{slot_id}",
        brief=brief_path,
        env={"OUTPUT_PATH": f"edit/animations/slot_{slot_id}/render.mp4"},
    )

# Example: launch three overlay agents in parallel

for slot in ["01", "02", "03"]:
    launch_overlay_agent(slot, f"edit/animations/slot_{slot}/brief.md")

Final Composition with FFmpeg

The render.py helper composites all parallel-generated assets, applying the PTS shift and ensuring subtitles are added last:

def build_final_composite(base, overlays, subtitles, out):
    # base = path to base video

    # overlays = list of dicts from the EDL

    # subtitles = optional SRT path

    filtergraph = "[0:v]"

    for idx, ov in enumerate(overlays, start=1):
        # shift overlay timing using PTS-STARTPTS+T/TB

        filtergraph += f"[{idx}:v]setpts=PTS-STARTPTS+{ov['start']}/TB"
        filtergraph += f"overlay=enable='between(t,{ov['start']},{ov['start']+ov['duration']})'[tmp{idx}]"
        filtergraph = f"[tmp{idx}]"

    # add subtitles last (if any)

    if subtitles:
        filtergraph += f"subtitles={subtitles}"
    
    # run ffmpeg

    ffmpeg_cmd = [
        "ffmpeg", "-i", base,
        *sum((["-i", ov["file"]] for ov in overlays), []),
        "-filter_complex", filtergraph,
        "-c:v", "libx264", "-crf", "18", out,
    ]
    subprocess.run(ffmpeg_cmd, check=True)

Summary

• Parallel Execution: Each overlay spawns an independent sub-agent via the Instagit Agent tool, executing concurrently to minimize wall-time.
• Sandboxed Briefs: Animation briefs are isolated in <edit>/animations/slot_<id>/ directories to prevent conflicts, as specified in SKILL.md.
• Framework Flexibility: Supports HyperFrames, Remotion, Manim, and PIL through framework-agnostic agent delegation.
• Precise Timing: FFmpeg setpts=PTS-STARTPTS+T/TB aligns overlay frame 0 with EDL window starts, compensating for variable render times.
• Layer Management: Subtitles are composited last to ensure visibility over opaque video overlays.

Frequently Asked Questions

How does video-use prevent filename collisions when spawning multiple sub-agents?

By writing each sub-agent's brief and output to distinct subdirectories under <edit>/animations/slot_<id>/, as defined in SKILL.md at line 81. Each slot operates in isolation, ensuring that concurrent writes never conflict regardless of how many animations run in parallel.

What animation frameworks does video-use support for overlay generation?

The repository supports HyperFrames, Remotion, Manim, and PIL (Python Imaging Library). The sub-agent receives the framework command in its brief and executes the appropriate rendering pipeline, allowing mixed-framework projects within a single EDL.

Why are subtitles added after overlays in the FFmpeg filter chain?

According to SKILL.md at line 22, subtitles must be applied last because opaque overlays would otherwise hide caption text. The rendering pipeline composites all video overlays first, then applies the subtitle filter to ensure maximum readability in the final output.

How does the parallel sub-agent approach affect total render time?

Because sub-agents run simultaneously in separate sandboxes, the total wall-time equals approximately the duration of the slowest individual animation rather than the sum of all animations. This makes the pipeline scalable for projects with many overlays, as confirmed by the "total wall time ≈ slowest one" rule in SKILL.md.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}