# Overlay PTS Shifting in video-use: Why Timestamps Must Be Rewritten for Proper Animation Timing

> Learn how overlay PTS shifting corrects video animation timing by rewriting timestamps to ensure accurate playback and prevent visual errors for viewers.

- Repository: [Browser Use/video-use](https://github.com/browser-use/video-use)
- Tags: deep-dive
- Published: 2026-08-06

---

**Overlay PTS shifting rewrites the presentation timestamps of animation overlays so their first frame aligns with their designated start time in the final output, preventing viewers from seeing an incorrect slice of the animation.**

In the `browser-use/video-use` pipeline, rendering a finished video requires compositing a **base clip** with **animation overlays**—short videos that play during specific windows in the output timeline. The `setpts` filter performs critical timestamp manipulation to synchronize these overlays correctly. This article explains the technical necessity of PTS shifting, its implementation in [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py), and the hard rule that governs this behavior.

## Why Overlay Timestamps Require Adjustment

Each overlay animation has its own internal timeline beginning at **PTS = 0**. When FFmpeg processes multiple inputs, it does not automatically align these timelines to the output video's clock. Without intervention, an overlay's first frame would appear at time 0 of the final render—regardless of when the overlay window actually opens.

This creates a **temporal misalignment problem**. Consider an overlay scheduled to play from **5.0 to 10.0 seconds** in the output. If its timestamps remain unshifted, the viewer sees **frame 0 at output time 0**, then sees **frame 5 seconds into the animation** when the overlay window finally opens. The result is jarring: users see the middle of the animation instead of its beginning.

## The PTS Shifting Formula

FFmpeg's `setpts` filter rewrites timestamps using expressions. For overlay alignment, `video-use` applies:

```

setpts=PTS-STARTPTS+<window_start>/TB

```

Breaking down this expression:

- `PTS-STARTPTS` — normalizes the stream so its first frame becomes PTS 0
- `<window_start>` — the overlay's start time in seconds (e.g., `5.000`)
- `TB` — timebase (inverse of frame rate), converting seconds to timestamp units

This shifts the overlay's entire timeline forward by its designated window start time. Frame 0 now appears exactly when the overlay window opens.

## Hard Rule 4: The Documentation Standard

The requirement for PTS shifting is codified as **Hard Rule 4** in the project's skill definition. From [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md):

> "Overlays use `setpts=PTS-STARTPTS+T/TB` to shift the overlay's frame 0 to its window start. Otherwise you see the middle of the animation during the overlay window."

This rule appears at lines 24-26 of the skill documentation, establishing PTS shifting as a non-negotiable step in the rendering pipeline.

## Implementation in helpers/render.py

The core rendering logic lives in [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py). For each overlay, the code constructs a `setpts` filter that incorporates the overlay's start offset:

```python

# Extracted from helpers/render.py, lines 21-25

for idx, ov in enumerate(overlays, start=1):
    t = float(ov["start_in_output"])
    # Shift overlay so its first frame aligns with its window start

    filter_parts.append(f"[{idx}:v]setpts=PTS-STARTPTS+{t}/TB[a{idx}]")

```

The loop processes overlays starting from index 1 (0 is reserved for the base video). Each shifted stream receives a labeled output (`[a1]`, `[a2]`, etc.) for subsequent overlay operations.

## Complete Filter Graph Assembly

The PTS-shifted streams feed into FFmpeg's `overlay` filter at precise time windows. Here is a representative command structure:

```bash
ffmpeg -i base.mp4 -i overlay1.mp4 -i overlay2.mp4 \
-filter_complex "[1:v]setpts=PTS-STARTPTS+5.000/TB[a1];
                 [2:v]setpts=PTS-STARTPTS+12.500/TB[a2];
                 [0:v][a1]overlay=enable='between(t,5.000,10.000)'[v1];
                 [v1][a2]overlay=enable='between(t,12.500,17.500)'[outv]" \
-map "[outv]" -map "0:a" -c:v libx264 final.mp4

```

Processing order matters critically:

1. **PTS shifting** (`setpts`) occurs first, retiming each overlay stream
2. **Compositing** (`overlay`) occurs second, applying enabled windows
3. **Final mapping** selects the composite video and original audio

## Rendering with EDL Files

Practical usage involves an EDL (Edit Decision List) JSON file specifying overlay windows:

```bash
python helpers/render.py edl.json -o final.mp4

```

The Python script parses timing metadata, computes `t` values for each overlay, and assembles the complete filter graph automatically.

## Summary

- **PTS shifting aligns independent timelines** — overlay animations and base video use different clocks that must be synchronized
- **`setpts=PTS-STARTPTS+t/TB` is the standard formula** — used in [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) for every overlay stream
- **Hard Rule 4 mandates this behavior** — documented in [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) as essential for correct visual output
- **Without shifting, viewers see animation midpoints** — the overlay window displays incorrect content from the animation's internal timeline

## Frequently Asked Questions

### What happens if I omit the PTS shift in video-use rendering?

The overlay plays from its internal time 0 at output time 0, regardless of when its window opens. When the overlay window finally activates, the viewer sees whatever frame corresponds to that moment in the animation's timeline—typically the middle or end rather than the beginning. This produces visually broken output where animations appear to start mid-action.

### How does `STARTPTS` differ from `PTS` in the filter expression?

`PTS` is the current frame's presentation timestamp. `STARTPTS` is the first frame's PTS of the stream. Subtracting yields a normalized timeline where the first frame equals zero. This normalization allows the `+t/TB` offset to work consistently regardless of the overlay's original timestamp values.

### Where is the overlay timing data defined in video-use?

Timing originates from transcription metadata generated by [`helpers/transcribe.py`](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py), which produces word-level timestamps used to schedule overlay windows. These values populate the `start_in_output` field in the overlay configuration, ultimately becoming the `t` parameter in the `setpts` expression within [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py).

### Can multiple overlays overlap in the same time window?

Yes. The filter graph chains overlays sequentially: `[0:v][a1]overlay` produces `[v1]`, then `[v1][a2]overlay` produces the next stage. Each overlay maintains independent PTS shifting via its designated `aN` label, allowing arbitrary overlap without timing interference.