Extraction Speed Differences Between Keyframe, Scene, and Uniform Frame Engines in Claude-Video

The keyframe-scene engine in claude-video extracts frames up to 10× faster than the uniform-frame engine by decoding only I-frames via -skip_frame nokey, while the uniform engine decodes the entire video stream using -vf fps=1.

The bradautomates/claude-video repository provides sophisticated video frame extraction capabilities through its watch skill. Understanding the extraction speed differences between keyframe, scene, and uniform frame engines is critical for optimizing video processing workflows. The extract_frames() function in skills/watch/scripts/frames.py implements two distinct strategies that trade off processing speed against temporal completeness.

How the Extraction Engines Work

Keyframe-Scene Engine (Fastest)

The keyframe-scene engine calls ffmpeg with the -skip_frame nokey option to extract only keyframes (I-frames) that begin each GOP (Group of Pictures). Because keyframes represent only 1%–10% of total frames in a typical video, the decoder skips all dependent frames entirely and jumps directly between keyframe positions.

This engine serves dual purposes: it extracts keyframes and effectively identifies scene changes, since I-frames typically occur at scene boundaries or significant visual changes. According to the implementation in skills/watch/scripts/frames.py, selecting engine="keyframe" triggers this fast extraction path.

Uniform-Frame Engine (Slower but Comprehensive)

The uniform-frame engine uses -vf fps=1 (or a user-specified frame rate) to extract frames at fixed temporal intervals regardless of video structure. This forces ffmpeg to decode the entire video stream sequentially, reconstructing every frame up to each target timestamp even when falling between keyframes.

As implemented in skills/watch/scripts/frames.py, passing engine="uniform" selects this approach, which provides regular temporal sampling but requires linearly increasing processing time as video length grows.

Performance Characteristics

Engine	FFmpeg Flags	Speed Impact	Best For
Keyframe-scene	-skip_frame nokey	Fastest – completes in seconds even for long videos	Quick scene summaries, thumbnail generation
Uniform-frame	-vf fps=N	Linear growth – 2–5× slower for 10-minute videos; scales with duration	Detailed temporal analysis, frame-per-second sampling

The speed gap exists because keyframe extraction leverages the video encoder's structure—accessing only self-contained I-frames—while uniform extraction requires full-frame decoding to reconstruct intermediate timestamps.

Implementation in the Source Code

In skills/watch/scripts/frames.py, the extract_frames() function selects the engine based on the engine parameter. When engine="keyframe", it invokes ffmpeg with -skip_frame nokey. When engine="uniform", it applies the fps filter with the specified interval.

The download() function in skills/watch/scripts/download.py handles video acquisition via yt-dlp, providing the source file path for extraction. The API contract documenting these parameters is defined in skills/watch/SKILL.md, which declares the optional engine and fps arguments for the /watch slash command.

Benchmarks for both engines are maintained in tests/test_frames.py, ensuring speed-related expectations remain consistent across versions.

Practical Usage Examples

Use the keyframe engine when you need rapid scene-level summaries:

from skills.watch.scripts.download import download
from skills.watch.scripts.frames import extract_frames

video_path = download("https://youtu.be/dQw4w9WgXcQ")
extract_frames(video_path, out_dir="keyframes", engine="keyframe")

Use the uniform engine when you require consistent temporal sampling for analysis:

from skills.watch.scripts.download import download
from skills.watch.scripts.frames import extract_frames

video_path = download("https://youtu.be/dQw4w9WgXcQ")
extract_frames(video_path, out_dir="uniform", engine="uniform", fps=1)

Both examples use the same public API; only the engine parameter changes to select the underlying ffmpeg strategy.

Summary

• Keyframe-scene engine uses -skip_frame nokey to achieve the fastest extraction speed by processing only I-frames, making it ideal for scene detection and thumbnail generation.
• Uniform-frame engine decodes the entire video stream using -vf fps= to provide regular temporal sampling, trading speed for comprehensive coverage.
• Choose the keyframe engine for quick visual summaries and the uniform engine for detailed frame-by-frame analysis.
• Both engines are available via extract_frames() in skills/watch/scripts/frames.py, with parameters documented in skills/watch/SKILL.md.

Frequently Asked Questions

What is the difference between keyframe and scene extraction in claude-video?

The keyframe engine effectively serves both purposes by extracting I-frames, which typically mark scene boundaries in encoded video. The repository implements this as a single engine="keyframe" option in skills/watch/scripts/frames.py rather than maintaining separate engines, since keyframes naturally indicate scene changes.

How much faster is the keyframe engine compared to uniform frame extraction?

The keyframe-scene engine is typically 2–5× faster for short videos and up to 10× faster for long videos, as it processes only 1%–10% of the total frame count by skipping dependent frames between keyframes, whereas the uniform engine must decode every frame sequentially.

Can I adjust the sampling rate for the uniform frame engine?

Yes, pass the fps parameter to extract_frames(). For example, fps=0.5 extracts one frame every two seconds, while fps=1 extracts one frame per second, as implemented in skills/watch/scripts/frames.py. Higher fps values increase extraction time linearly.

Where are the extraction engines tested in the codebase?

The test suite in tests/test_frames.py benchmarks both engines to ensure speed expectations and output correctness are maintained across versions, validating that the keyframe engine remains significantly faster than the uniform approach.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}