How Timestamp Cue Frames Work with the `--timestamps` Option in Claude Video

The --timestamps option parses comma-separated timecodes into pinned cue frames that are extracted via ffmpeg and merged with automatic frame selections, guaranteeing these exact moments always appear in the final output regardless of sampling limits.

Claude Video is an open-source video analysis framework that combines automatic frame sampling with user-directed extraction. When you use the --timestamps flag, the system converts human-readable time strings—like "1,3:15,00:02:30"—into deterministic timestamp cue frames that bypass the standard detail engine's heuristics.

Parsing Timestamp Strings into Seconds

The process begins in skills/watch/scripts/frames.py, where the parse_timestamps function handles raw user input.

The parse_timestamps Function

Located at lines 95–110 in frames.py, this utility converts a comma-separated string into a sorted, de-duplicated list of seconds:

• Each token is stripped and converted using parse_time, which understands SS, MM:SS, and HH:MM:SS formats
• Invalid tokens raise an error immediately
• The output is sorted numerically to ensure chronological extraction order

from skills.watch.scripts.frames import parse_timestamps

# Returns [1.0, 150.0, 150.0] → deduplicated to [1.0, 150.0]

timestamps = parse_timestamps("1,2:30,00:02:30")

Extracting Cue Frames at Specific Timestamps

Once parsed, the timestamps flow to extract_at_timestamps (lines 24–41 in frames.py), which manages the actual frame generation pipeline.

Cleaning Existing Cues

The function first removes any stale cue_*.jpg files from the output directory to prevent collisions with previous runs.

Window Filtering and Sampling Constraints

Before extraction, the system applies user-specified boundaries:

1. Focus window filtering: Timestamps outside the start_seconds/end_seconds range are dropped from consideration
2. Max frames enforcement: If the surviving timestamp count exceeds --max-frames, the _even_indices helper evenly samples the list while preserving the first and last entries

FFmpeg Single-Frame Extraction

For each valid timestamp, the system executes a targeted ffmpeg command:

ffmpeg -ss <timestamp> -i input.mp4 -frames:v 1 cue_<n>.jpg

• The -ss parameter seeks to the exact second (accurate to milliseconds)
• -frames:v 1 extracts exactly one frame
• Output is saved as cue_<n>.jpg in the designated directory

Metadata about the operation—including the engine type ("timestamps"), candidate counts, and any dropped timestamps—returns to the caller for logging.

Merging Cue Frames with Automatic Selection

After extraction, merge_frames (lines 12–22 in frames.py) integrates the cue frames with frames generated by the primary detail engine (scene detection, uniform sampling, etc.).

The Pinning Mechanism

Cue frames are treated as pinned content, meaning they receive priority protection during budget enforcement. When the total frame count exceeds limits:

1. Cue frames are merged first
2. The frame budget cap is applied only to the automatic selection pool
3. User-requested timestamps are never evicted, even when automatic frames must be discarded

This ensures deterministic coverage of critical moments while allowing the detail engine to fill remaining slots with contextually relevant frames.

CLI Integration in watch.py

The orchestration occurs in skills/watch/scripts/watch.py:

• Parsing: Lines 81–82 call parse_timestamps(args.timestamps) to validate input
• Conditional extraction: Lines 76–88 check if timestamps exist and invoke extract_at_timestamps only when a video file is present
• Reporting: Lines 304–306 include the cue count in the final summary output

# Extract specific moments alongside balanced auto-sampling

watch "https://example.com/video.mp4" \
  --timestamps "1,3:15,00:02:30" \
  --detail balanced \
  --max-frames 10

Summary

• parse_timestamps converts human timecodes (1:05, 00:02:30) into numeric seconds and removes duplicates
• extract_at_timestamps filters timestamps against focus windows, samples if exceeding --max-frames, and extracts JPEGs via ffmpeg
• Cue frames are pinned during the merge process, guaranteeing they survive frame budget limitations
• The workflow is coordinated through watch.py, which handles CLI arguments and reporting

Frequently Asked Questions

What timestamp formats does Claude Video support?

Claude Video accepts three formats: bare seconds (90), minutes and seconds (1:30), and full timestamps (00:01:30). The parse_time helper in frames.py automatically detects the format and converts to seconds since the video start.

How does the --max-frames option affect timestamp cue frames?

When the number of requested timestamps exceeds --max-frames, the system uses _even_indices to evenly sample the timestamp list while always keeping the first and last requested moments. This maintains user intent while respecting computational limits.

Can cue frames be evicted if the total frame budget is limited?

No. According to the merge_frames implementation in frames.py, cue frames are pinned and merged before the frame budget cap is applied. The automatic detail engine's output is reduced to accommodate the cap, but user-requested timestamp frames are never removed.

Where does the actual frame extraction happen in the codebase?

The ffmpeg execution and file I/O occur in extract_at_timestamps within skills/watch/scripts/frames.py (lines 24–41). This function is called from the CLI entry point in skills/watch/scripts/watch.py when the --timestamps argument is present.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}