# How Keyframe Extraction (--detail efficient) Works Compared to Scene Detection

> Learn how keyframe extraction uses I-frame skipping for speed while scene detection decodes full video for precise cuts. Compare efficiency and granularity for your video analysis needs.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-08-06

---

**Keyframe extraction (--detail efficient) uses ffmpeg's I-frame skipping for fast, low-cost sampling, while scene detection (--detail scene) performs full video decoding to find visual cuts, trading speed for granularity.**

In the `bradautomates/claude-video` repository, the `watch` skill provides multiple strategies for extracting representative frames from video files. Understanding the difference between `--detail efficient` and `--detail scene` helps you choose the right balance between processing speed and visual granularity when analyzing video content.

## What Is Keyframe Extraction (--detail efficient)?

The `--detail efficient` mode extracts only **keyframes** (I-frames) already embedded in the video stream. These frames represent complete images that don't depend on other frames for decoding, making them natural candidates for video summarization.

### Implementation Details

The core logic resides in `extract_keyframes()` within **[`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py)** (lines 1006–1014). This function invokes ffmpeg with the `-skip_frame nokey` flag, which instructs the decoder to skip all non-keyframes and extract only I-frames:

```python

# From frames.py - keyframe extraction logic

ffmpeg -skip_frame nokey -i input.mp4 ...

```

After extraction, timestamps are parsed from ffmpeg's `showinfo` output. The pipeline then applies `dedupe_perceptual()` to remove near-identical frames and optionally applies `_even_sample()` to respect the `max_frames` cap while ensuring the first and last frames are always preserved.

### Fallback Behavior

If fewer than `KEYFRAME_MIN` (approximately 4) keyframes are detected, the engine automatically falls back to uniform-budget extraction by calling `extract()` (lines 1036–1069). This ensures you always receive a usable set of frames even in videos with sparse keyframe placement.

## What Is Scene Detection (--detail scene)?

The `--detail scene` mode identifies **scene cuts**—points where visual content changes significantly between consecutive frames—regardless of whether those frames are keyframes.

### Implementation Details

Implemented in `extract_scene_candidates()` within **[`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py)** (lines 1052–1056), this method uses ffmpeg's `select` filter with a scene detection threshold:

```bash
ffmpeg -i input.mp4 -vf "select='eq(n\,0)+gt(scene\,THRESHOLD)'" ...

```

This forces a **full decode** of the entire video, computing the scene change metric for every frame. When the metric exceeds `SCENE_THRESHOLD` (default 0.20), ffmpeg emits that frame. The first frame is always included to ensure coverage from the video start.

### Processing Overhead

Unlike keyframe extraction, scene detection requires decoding every frame to calculate visual differences. This results in significantly higher CPU usage and longer processing times, making it more expensive for large video files or batch processing.

### Fallback Behavior

If the detected scene count falls below `SCENE_MIN_FRAMES` (approximately 8), the system falls back to uniform sampling via `extract()` (lines 1050–1054). This prevents under-sampling in videos with minimal visual changes.

## Key Differences and Performance Implications

**Speed vs. Granularity**: `--detail efficient` is ideal for quick previews or processing many videos, as it only touches the keyframe index table. `--detail scene` yields richer frame sets for videos with subtle cuts but requires full-video decoding.

**Frame Distribution**: Keyframe extraction relies on encoder decisions about where to place I-frames, which typically occur at natural cut points but may miss rapid visual transitions. Scene detection captures any significant visual change, potentially providing more granular coverage.

**Resource Consumption**: Keyframe extraction is **very cheap**—ffmpeg seeks directly to keyframe locations without full decoding. Scene detection is **more expensive**, requiring complete frame processing to compute the scene metric.

## Practical Usage Examples

Both modes respect the `--max-frames` parameter and apply perceptual deduplication before returning results. The entry point in **[`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py)** parses the `--detail` flag and dispatches to the appropriate extraction function.

**Efficient keyframe extraction** for quick analysis:

```bash
python -m skills.watch.scripts.frames \
    /path/to/video.mp4 /tmp/out \
    --detail efficient \
    --max-frames 50

```

**Scene-change detection** for detailed visual analysis:

```bash
python -m skills.watch.scripts.frames \
    /path/to/video.mp4 /tmp/out \
    --detail scene \
    --max-frames 80

```

## Summary

- **Keyframe extraction** (`--detail efficient`) uses `ffmpeg -skip_frame nokey` to extract only I-frames, offering fast, low-cost processing with minimal CPU overhead.
- **Scene detection** (`--detail scene`) uses ffmpeg's `select` filter with a scene threshold to find visual cuts, requiring full-video decoding but providing finer granularity.
- Both modes apply `dedupe_perceptual()` and `_even_sample()` to remove duplicates and respect `max_frames` limits.
- Fallback mechanisms ensure usability: `KEYFRAME_MIN` (4) for efficient mode and `SCENE_MIN_FRAMES` (8) for scene mode trigger uniform sampling if insufficient frames are detected.
- The implementation resides in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), with CLI handling in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py).

## Frequently Asked Questions

### When should I use `--detail efficient` versus `--detail scene`?

Use `--detail efficient` when processing speed and low resource usage are priorities, such as when analyzing many videos or generating quick previews. Use `--detail scene` when you need to capture subtle visual transitions or rapid cuts that might occur between keyframes, and can afford the additional processing time.

### How does the fallback mechanism work?

According to the source code in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), if keyframe extraction finds fewer than 4 frames (`KEYFRAME_MIN`), or scene detection finds fewer than 8 frames (`SCENE_MIN_FRAMES`), the system automatically switches to uniform sampling via `extract()`. This ensures you always receive a representative set of frames even when the primary extraction strategy yields sparse results.

### What is perceptual deduplication?

After initial frame extraction, both modes call `dedupe_perceptual()` to remove visually similar frames that might represent the same scene. This prevents redundant analysis of nearly identical images before the final `_even_sample()` step distributes frames evenly across the video timeline.

### Can I control the number of extracted frames?

Yes. Both extraction modes respect the `--max-frames` parameter passed via the CLI in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py). After deduplication, the `_even_sample()` function ensures the final frame count doesn't exceed your specified maximum while preserving the first and last frames of the video for complete temporal coverage.