# How Animation Payoff Timing and Narration Sync Work in video-use

> Discover how video-use synchronizes overlay animations with narration. Learn about animation payoff timing and sync strategies for precise visual storytelling.

- Repository: [Browser Use/video-use](https://github.com/browser-use/video-use)
- Tags: internals
- Published: 2026-06-29

---

**`video-use` synchronizes overlay animations to spoken narration by starting animations `reveal_duration` seconds before a payoff word timestamp, ensuring the final frame lands exactly when the key phrase is spoken.**

The `browser-use/video-use` repository treats narration as the master clock for all visual overlays. This approach ensures that animated diagrams, text cards, and complex visuals reinforce the spoken narrative at precisely the right moment. Understanding how **animation payoff timing and narration sync** interact is essential for producing professional-grade educational content where visuals and audio feel seamlessly integrated.

## The Two Core Synchronization Rules

The synchronization strategy defined in [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) relies on two complementary timing constraints that govern how overlays appear relative to the spoken track.

### Sync-to-Narration Timing

Every narration-driven overlay must persist long enough for viewers to comfortably read and comprehend the content. According to the guidelines in [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) at line 218, durations follow these floors:

- **3 seconds** – absolute minimum for any readable overlay
- **5–7 seconds** – typical duration for simple text cards
- **8–14 seconds** – required for complex diagrams with multiple elements

This rule ensures accessibility and prevents cognitive overload, regardless of when the animation technically begins.

### Animation Payoff Timing

The **payoff** refers to the final frame of an animation—the moment when the complete diagram or text block is fully revealed. To maximize impact, this payoff frame must coincide exactly with the spoken **payoff word** (the key term or concept being explained).

As implemented in [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) at line 224, the overlay start time is calculated by subtracting the animation's duration from the payoff word's timestamp:

```python
start_in_output = payoff_timestamp - reveal_duration

```

This guarantees that after `reveal_duration` seconds of animation (e.g., drawing, fading, or building elements), the viewer sees the completed visual exactly when the narrator utters the critical phrase.

## Implementation in the Rendering Pipeline

The actual timestamp alignment happens in [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py), which processes the Edit Decision List (EDL) and constructs the final FFmpeg command.

### PTS Shifting for Frame Alignment

When [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) processes overlay segments (lines 7–9), it applies a **PTS shift** using the `setpts` filter:

```bash
setpts=PTS-STARTPTS+T/TB

```

This formula adjusts the Presentation Timestamp so that frame 0 of the overlay video aligns with the calculated `start_in_output` value in the EDL timeline. The overlay essentially becomes a contiguous stream that starts playing at the precise moment needed to hit the payoff word.

### Filter Graph Ordering

The rendering pipeline applies the subtitle filter **last** to ensure that captions remain visible above any animated overlays. This prevents animation elements from obscuring text that viewers need to read.

## Calculating Overlay Start Times in Practice

To implement payoff timing in your own EDL workflows, use the following pattern when building overlay entries. This Python snippet demonstrates how to compute the correct `start_in_output` value given a payoff word timestamp and desired reveal duration:

```python
import json
from pathlib import Path

# --------------------------------------------------------------

# Helper: compute overlay start based on payoff word timestamp

# --------------------------------------------------------------

def make_overlay(entry, payoff_ts: float, reveal_dur: float) -> dict:
    """
    entry: dict from the EDL's `overlays` list (may contain file path, duration)
    payoff_ts: timestamp of the spoken payoff word (seconds)
    reveal_dur: how many seconds before the payoff the animation should start
    """
    start = max(0.0, payoff_ts - reveal_dur)          # never start before 0

    entry["start_in_output"] = round(start, 3)       # precision matches ffprobe output

    entry["duration"] = round(reveal_dur + 0.5, 3)   # include a short tail after payoff

    return entry

# --------------------------------------------------------------

# Example usage – building an EDL for a simple card overlay

# --------------------------------------------------------------

edl_path = Path("edit/edl.json")
edl = json.loads(edl_path.read_text())

# Suppose we have an overlay video already rendered at `edit/animations/slot_3/render.mp4`

overlay = {
    "file": "edit/animations/slot_3/render.mp4",
    "duration": 0,           # placeholder – will be overwritten

    "start_in_output": 0,    # placeholder – will be overwritten

}

# Payoff word occurs at 12.34 s in the narration; we want a 2-second reveal

payoff_timestamp = 12.34
reveal_duration = 2.0

edl["overlays"].append(make_overlay(overlay, payoff_timestamp, reveal_duration))

# Save the modified EDL

edl_path.write_text(json.dumps(edl, indent=2))
print("EDL updated with payoff-timed overlay.")

```

When [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) processes this EDL, it shifts the overlay's PTS so the animation begins at 10.34 seconds (12.34 - 2.0), revealing for exactly 2 seconds until the payoff word at 12.34 seconds. The optional 0.5-second tail keeps the final frame visible briefly after the narrator finishes the key phrase.

## Summary

- **Animation payoff timing** in `video-use` requires calculating `start_in_output = payoff_timestamp - reveal_duration` to align final animation frames with spoken keywords.
- **Sync-to-narration** rules mandate minimum durations of 3 seconds (floor), with 5–7 seconds for simple content and 8–14 seconds for complex diagrams per [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md).
- The rendering pipeline in [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) uses FFmpeg's `setpts=PTS-STARTPTS+T/TB` to align overlay frame 0 with EDL start times.
- Subtitle filters are applied last in the filter chain to prevent overlays from hiding captions.
- EDL files stored at [`edit/edl.json`](https://github.com/browser-use/video-use/blob/main/edit/edl.json) serve as the bridge between narrative timestamps and precise animation timing.

## Frequently Asked Questions

### What is a "payoff word" in the context of video-use?

A **payoff word** is the specific term or concept in the narration that represents the key learning moment for a given visual. In `video-use`, animations are timed so that their final "payoff" frame—showing the completed diagram or text—appears exactly when the narrator speaks this word, creating immediate visual-audio reinforcement.

### How long should narration-driven overlays remain on screen?

According to [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md), overlays must last at least **3 seconds** to be readable, with **5–7 seconds** being typical for simple cards and **8–14 seconds** for complex diagrams. These durations are independent of animation length; if an animation takes 2 seconds to reveal, the total overlay time must still meet the minimum readability threshold.

### Why does the subtitle filter run last in the rendering pipeline?

The rendering pipeline in [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) applies the subtitle filter **after** all video and overlay filters to ensure that captions remain visible on top of any animated content. This ordering prevents complex diagrams or visual effects from obscuring text that viewers need to read.

### How do I adjust reveal duration for complex Manim animations?

For Manim-based diagrams (referenced in [`skills/manim-video/SKILL.md`](https://github.com/browser-use/video-use/blob/main/skills/manim-video/SKILL.md)), calculate the `reveal_duration` to match the actual render time of the build sequence, then add a small buffer (0.5–1.0 seconds) to the total overlay duration. This ensures the diagram finishes constructing exactly at the payoff word while remaining visible long enough for the viewer to study the final result.