# Scene-Change Detection vs Keyframe Extraction in Claude-Video: Technical Differences Explained

> Discover the technical differences between scene-change detection and keyframe extraction in Claude-Video. Optimize your video analysis workflow with accurate insights.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: technical-differences
- Published: 2026-07-25

---

**Scene-change detection performs pixel-level analysis across every decoded frame to identify visual transitions, while keyframe extraction rapidly selects only pre-existing I-frames already embedded by the video encoder, trading computational accuracy for processing speed.**

Choosing the right frame extraction strategy in `bradautomates/claude-video` directly impacts both the quality of your video analysis and processing overhead. The repository's `frames` module, located at [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), implements these two approaches through distinct ffmpeg-based pipelines designed for different content types and performance requirements.

## How Keyframe Extraction Works

### The I-Frame Selection Pipeline

The `extract_keyframes()` function operates by invoking ffmpeg with the `-skip_frame nokey` flag, which instructs the decoder to process only **I-frames** (keyframes) already present in the video stream. These frames are inserted by the encoder at scene cuts or regular intervals during the original compression.

Implemented in lines [776-885](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py#L776-L885) of [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), this method optionally deduplicates near-identical frames and applies even-sampling to respect a `max_frames` limit. Because ffmpeg skips non-key frames entirely without decoding them, this approach completes in milliseconds for most files, making it ideal for quick previews when the video already contains sufficient encoder-generated cuts.

## How Scene-Change Detection Works

### Content-Aware Frame Analysis

In contrast, `extract_scene_candidates()` (lines [174-226](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py#L174-L226)) runs ffmpeg with a *select filter* that evaluates the `scene` metric using the expression `gt(scene, threshold)`. Each time the per-frame pixel difference exceeds the configured threshold—defaulting to **0.20** (`SCENE_THRESHOLD`)—ffmpeg emits that frame as a scene boundary.

This method forces ffmpeg to decode every frame in the video, analyzing actual visual content rather than relying on encoder metadata. The function always retains the first frame regardless of threshold, then applies deduplication and sampling to the results. This approach excels with content that has few or irregular keyframes, such as screen recordings or static slide presentations.

## Critical Differences Between Approaches

Understanding the technical distinctions helps you select the appropriate method for your specific use case:

- **Data Source**: **Keyframe extraction** uses only existing I-frames baked into the video stream, while **scene-change detection** recomputes cuts by analyzing frame-to-frame pixel differences.
- **Processing Cost**: Keyframe extraction avoids full decoding, consuming minimal CPU. Scene detection decodes every frame, creating significant computational overhead but yielding finer-grained coverage.
- **Encoder Independence**: If a video contains few keyframes (common in long screen recordings), keyframe extraction may return insufficient frames. Scene detection works regardless of encoder settings because it evaluates actual pixel changes.
- **Configurability**: Scene detection exposes a `threshold` parameter for tuning sensitivity to motion, while keyframe extraction offers no comparable content-based parameter.

## The Fallback Strategy: extract_scene_or_uniform()

The library implements intelligent fallback logic in `extract_scene_or_uniform()` (lines [511-574](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py#L511-L574)). This function first attempts scene detection; if fewer than `SCENE_MIN_FRAMES` are found, it automatically falls back to uniform sampling across the timeline.

This ensures robust behavior for static recordings where scene detection would return almost nothing, guaranteeing a usable set of representative frames even when visual content changes minimally over time.

## Practical Implementation Examples

Both extraction methods follow a consistent API pattern, returning lists of frame metadata dictionaries containing `index`, `timestamp_seconds`, `path`, and `reason` fields:

```python
from pathlib import Path
from skills.watch.scripts.frames import extract_keyframes, extract_scene_candidates

video = "example.mp4"
out_dir = Path("frames")

# Strategy 1: Keyframe extraction (fast, encoder-dependent)

keyframes, key_meta = extract_keyframes(
    video_path=video,
    out_dir=out_dir,
    resolution=512,
    max_frames=50,
    dedup=True,
)
print(f"Engine: {key_meta['engine']} | Found: {len(keyframes)} frames")

# Strategy 2: Scene-change detection (slower, content-aware)

scene_frames, scene_meta = extract_scene_candidates(
    video_path=video,
    out_dir=out_dir,
    resolution=512,
    max_frames=100,
    threshold=0.20,  # Adjust sensitivity (0.0-1.0)

)
print(f"Engine: {scene_meta['engine']} | Found: {len(scene_frames)} frames")

```

## Integration with the Processing Pipeline

The [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) file orchestrates the end-to-end workflow, selecting the appropriate extraction engine based on video characteristics. While [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) handles metadata collection and fps calculation, the underlying frame extraction logic remains contained within [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), making these functions reusable for standalone frame processing tasks separate from the main pipeline.

## Summary

- **Keyframe extraction** in `claude-video` leverages existing I-frames via ffmpeg's `-skip_frame nokey` for maximum speed, defined in `extract_keyframes()` at lines 776-885 of [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py).
- **Scene-change detection** analyzes pixel differences using ffmpeg's select filter with a configurable threshold (default 0.20), implemented in `extract_scene_candidates()` at lines 174-226.
- Scene detection requires full frame decoding, making it slower but more reliable for videos with sparse or irregular keyframes.
- The `extract_scene_or_uniform()` function (lines 511-574) provides automatic fallback to uniform sampling when scene detection yields fewer than `SCENE_MIN_FRAMES`.
- Both methods return standardized metadata dictionaries compatible with downstream captioning and thumbnail generation workflows.

## Frequently Asked Questions

### Which method should I use for screen recordings?

Use scene-change detection. Screen recordings often contain long static sections with few keyframes, making keyframe extraction return an insufficient number of frames. The `extract_scene_candidates()` function analyzes actual pixel changes to detect meaningful transitions regardless of encoder settings.

### Can I adjust the sensitivity of scene detection?

Yes. The `threshold` parameter in `extract_scene_candidates()` accepts values between 0.0 and 1.0, with the default set to 0.20 (`SCENE_THRESHOLD`). Lower values detect subtle transitions, while higher values only capture major scene cuts.

### Why does keyframe extraction run faster than scene detection?

Keyframe extraction uses ffmpeg's `-skip_frame nokey` flag to decode only I-frames, skipping all intermediate frames. Scene detection requires decoding every frame to calculate pixel differences, which consumes significantly more CPU cycles but provides content-aware accuracy.

### What happens if a video has no scene changes?

The `extract_scene_or_uniform()` function automatically falls back to uniform temporal sampling when fewer than `SCENE_MIN_FRAMES` are detected, ensuring you always receive a usable set of representative frames spread across the video timeline.