PTS-STARTPTS Overlay Shifting Technique: Precise Video Compositing with FFmpeg

The PTS-STARTPTS overlay shifting technique resets each overlay's timeline to zero and offsets it to the exact start time, ensuring animations begin at their first frame rather than mid-stream when compositing multiple video layers.

When compositing multiple video streams in FFmpeg, each overlay maintains its own internal timeline starting at timestamp zero. The video-use repository implements a robust PTS-STARTPTS overlay shifting technique to prevent mid-animation artifacts and ensure seamless picture-in-picture rendering across all layers.

What is the PTS-STARTPTS Overlay Shifting Technique?

The PTS-STARTPTS overlay shifting technique is an FFmpeg filter expression that synchronizes overlay streams with their intended display windows in a composite video. Without this adjustment, an overlay enabled at 5 seconds into the final video would display the frame at its own 5-second mark rather than its first frame, causing the animation to begin mid-stream.

The technique employs the setpts filter with the following expression:

[<idx>:v]setpts=PTS-STARTPTS+<T>/TB[a<idx>]

This expression performs two critical operations:

• PTS-STARTPTS – Subtracts the overlay's initial presentation timestamp, effectively resetting its internal timeline to zero and normalizing the stream to start at frame 0.
• +/TB – Adds the desired start offset T (in seconds) expressed in time-base units (TB), positioning the reset timeline at the exact moment the overlay should appear in the output.

The result is a labeled stream a<idx> whose first frame appears precisely when the overlay window opens.

Why the PTS-STARTPTS Shift is Required for Video Compositing

According to the Hard Rules in SKILL.md (line 25), this technique is mandatory for three specific reasons:

• Correct visual timing – Ensures the overlay begins with its first frame, not a random later frame from its internal timeline.
• Predictable compositing – The overlay's duration and enable window, calculated using between(t,...), reference the exact start time (T) relative to the base video timeline.
• Avoids visual artifacts – Without the shift, viewers would observe a "jump" or partially-played animation during the overlay period, breaking the seamless visual experience.

Implementation in the video-use Repository

The video-use repository implements this technique in helpers/render.py (lines 22-25), where the filter complex is constructed programmatically.

Building the PTS-Shift Filter

The following Python code from helpers/render.py constructs the setpts filter for each overlay:

# Build filter parts for each overlay

for idx, ov in enumerate(overlays, start=1):
    t = float(ov["start_in_output"])
    # Reset overlay timeline to 0 and offset to overlay start time

    filter_parts.append(f"[{idx}:v]setpts=PTS-STARTPTS+{t}/TB[a{idx}]")

Here, ov["start_in_output"] represents the time in seconds where the overlay should appear in the final video. The expression creates the shifted stream label a{idx}.

Chaining Overlays with Shifted Streams

After creating the PTS-shifted streams, the code chains them onto the base video using the overlay filter:

# Chain overlays on top of the base video

current = "[0:v]"
for idx, ov in enumerate(overlays, start=1):
    t = float(ov["start_in_output"])
    dur = float(ov["duration"])
    end = t + dur
    next_label = f"[v{idx}]"
    filter_parts.append(
        f"{current}[a{idx}]overlay=enable='between(t,{t:.3f},{end:.3f})'{next_label}"
    )
    current = next_label

In this pipeline, [a{idx}] references the previously shifted overlay stream, and the overlay filter activates only between the calculated start (t) and end (end) times.

Complete FFmpeg Command Example

The following simplified command demonstrates the complete filter complex for two overlays starting at 5 and 12 seconds:

ffmpeg -y -i base.mp4 -i overlay1.mp4 -i overlay2.mp4 \
  -filter_complex "\
    [1:v]setpts=PTS-STARTPTS+5/TB[a1]; \
    [2:v]setpts=PTS-STARTPTS+12/TB[a2]; \
    [0:v][a1]overlay=enable='between(t,5,10)'[v1]; \
    [v1][a2]overlay=enable='between(t,12,17)'[outv]" \
  -map "[outv]" -map 0:a -c:v libx264 -preset fast -crf 18 output.mp4

Overlay 1 starts at 5 seconds (duration 5 seconds), and overlay 2 starts at 12 seconds (duration 5 seconds). The setpts calls guarantee each overlay begins at its first frame exactly when the overlay filter becomes active.

Summary

• PTS-STARTPTS resets the overlay's internal timeline to zero, ensuring frame 0 aligns with the overlay's intended start time in the final output.
• The +/TB component converts the desired start time into time-base units for precise positioning within the FFmpeg filter graph.
• The technique is implemented in helpers/render.py (lines 22-25) and mandated by the Hard Rules in SKILL.md (line 25).
• Without this shift, overlays would display mid-animation, causing visual jumps and incorrect timing in the final composite.

Frequently Asked Questions

What does PTS-STARTPTS mean in FFmpeg?

PTS-STARTPTS subtracts the presentation timestamp of the first frame from every subsequent timestamp in the stream. This operation resets the video's timeline to zero, making frame 0 the new reference point regardless of when the clip was extracted or its original timestamp values.

Why can't I just use the overlay filter without shifting timestamps?

Without the PTS-STARTPTS shift, an overlay enabled at 10 seconds in the final output would display the frame at 10 seconds of the overlay's internal timeline. Since overlays are typically short clips meant to play from their beginning, this would show a mid-animation frame instead of the intended opening frame, creating a jarring visual discontinuity.

What is the TB variable in the setpts expression?

TB represents the time base of the video stream, which is the inverse of the frame rate (1/fps). FFmpeg requires time offsets in the setpts filter to be expressed in time-base units rather than seconds. Dividing the seconds value T by TB converts the seconds-based start time into the correct units for the filter calculation.

Where is this technique documented in the video-use repository?

The technique is documented as a Hard Rule in SKILL.md at line 25, which states that every overlay must have its PTS shifted to align frame 0 with the overlay window. The implementation resides in helpers/render.py (lines 22-25), where the filter expression is constructed programmatically for the FFmpeg command pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}