# extract_keyframes vs extract_scene_or_uniform in Claude Video: Efficient vs Balanced Frame Extraction

> Compare extract_keyframes for efficient I-frame extraction vs extract_scene_or_uniform for balanced scene-cut detection in Claude Video. Learn which method suits your needs.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: internals
- Published: 2026-07-30

---

**In the bradautomates/claude-video repository, `extract_keyframes` provides near-instant extraction by sampling only I-frames using ffmpeg's `-skip_frame nokey`, while `extract_scene_or_uniform` performs computationally expensive scene-cut detection across the entire video stream to identify meaningful transitions, falling back to uniform sampling for static content.**

The `watch` skill in bradautomates/claude-video provides two primary video frame extraction strategies that trade speed against semantic coverage. Understanding the architectural differences between these functions allows you to optimize the `--detail` flag for your specific latency and accuracy requirements.

## How the Two Functions Differ Internally

Both functions reside in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) and serve different operational tiers of the video processing pipeline. Their implementation reveals a fundamental trade-off between decoder efficiency and visual comprehension.

### extract_keyframes: Fast Keyframe-Only Extraction

The `extract_keyframes` function (defined at line 576 in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py)) optimizes for speed by leveraging existing encoder metadata rather than analyzing visual content.

The implementation follows this sequence:

1. Invokes **ffmpeg** with the `-skip_frame nokey` flag, instructing the decoder to reconstruct only I-frames (keyframes) that the encoder previously inserted at scene boundaries.
2. Parses ffmpeg's `showinfo` output to collect precise timestamps for each keyframe.
3. Checks against `KEYFRAME_MIN`: if the clip contains fewer keyframes than this threshold, it falls back to a uniform sampler that performs full decoding.
4. De-duplicates near-identical frames when `dedup=True`, then evenly samples the remaining candidates to meet `max_frames` (defaulting to **50** for efficient mode).

This path never decodes every frame, resulting in processing times of approximately **0.5 seconds** per clip, making it the fastest extraction tier.

### extract_scene_or_uniform: Scene-Aware Detection with Fallback

The `extract_scene_or_uniform` function (defined at line 510 in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py)) prioritizes semantic coverage by analyzing actual visual changes across the complete video stream.

The algorithm executes as follows:

1. Runs `extract_scene_candidates` (a scene-cut detector) over the **entire decoded clip**, identifying genuine visual transitions rather than relying on encoder keyframes.
2. Validates against `SCENE_MIN_FRAMES`: if detected cuts exceed this threshold, de-duplicates frames and evenly samples to the configured `max_frames` (defaulting to **100** for balanced mode).
3. Falls back to `extract` (uniform sampling at calculated fps) when the video lacks sufficient scene changes, ensuring coverage for static content.
4. Returns metadata indicating `"scene"` when cuts are detected or `"uniform"` when falling back.

Because this function must decode every frame to detect cuts, it incurs significantly higher computational costs than the keyframe-only approach, but provides semantically meaningful frame selection for most video content.

## Performance and Coverage Comparison

The `watch` skill maps these functions to CLI detail flags in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) (lines 198–208):

| Detail Flag | Function Called | Default Frame Cap | Decode Strategy | Typical Use Case |
|-------------|-----------------|-------------------|-----------------|------------------|
| `efficient` | `extract_keyframes` | 50 | Keyframe-only (I-frames) | High-volume processing, speed-critical |
| `balanced` | `extract_scene_or_uniform` | 100 | Full decode with scene detection | General analysis, semantic accuracy |

**Key distinction:** The efficient path relies on **encoder-inserted keyframes**, which may not align perfectly with actual scene changes if the encoder placed I-frames for structural rather than visual reasons. The balanced path performs **content-aware analysis**, ensuring frames capture genuine visual transitions even when keyframes are sparse.

## Practical Code Examples

### Efficient Mode Implementation

```python
from pathlib import Path
from skills.watch.scripts.frames import extract_keyframes

# Fast extraction using keyframe-only sampling (≤50 frames)

frames, meta = extract_keyframes(
    video_path="interview_clip.mp4",
    out_dir=Path("/tmp/frames"),
    max_frames=50,      # Default cap for efficient mode

    dedup=True,
)

print(meta["engine"])   # Outputs: "keyframe"

print(len(frames))      # ≤ 50 frames

```

### Balanced Mode Implementation

```python
from pathlib import Path
from skills.watch.scripts.frames import extract_scene_or_uniform

# Scene-aware extraction with uniform fallback (≤100 frames)

frames, meta = extract_scene_or_uniform(
    video_path="interview_clip.mp4",
    out_dir=Path("/tmp/frames"),
    target_frames=100,  # Requested frame count

    max_frames=100,     # Hard cap for balanced mode

    dedup=True,
)

print(meta["engine"])   # Outputs: "scene" or "uniform" if fallback triggered

print(len(frames))      # ≤ 100 frames

```

## Summary

- **`extract_keyframes`** provides **speed** by extracting only existing I-frames via ffmpeg's `-skip_frame nokey`, processing clips in ~0.5 seconds with a default cap of 50 frames.
- **`extract_scene_or_uniform`** provides **accuracy** by decoding the full stream to detect genuine scene cuts, supporting up to 100 frames with intelligent fallback to uniform sampling for static videos.
- Both functions support **de-duplication** and **even-sampling** to meet frame caps while avoiding redundant visual information.
- The choice between `efficient` and `balanced` modes in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) determines whether your pipeline prioritizes latency or semantic coverage.

## Frequently Asked Questions

### When should I use efficient mode versus balanced mode in Claude Video?

Use **efficient** mode (`extract_keyframes`) when processing large video archives where latency matters more than precise scene alignment, as it completes in under a second by reading only pre-encoded I-frames. Use **balanced** mode (`extract_scene_or_uniform`) when analyzing content where missing a scene transition would reduce comprehension, such as educational videos or dialogue-heavy content, accepting the full-decode performance cost for better semantic coverage.

### Why does balanced mode require full video decoding while efficient mode does not?

The `extract_scene_or_uniform` function must decode every frame to calculate visual differences between consecutive images and identify genuine scene cuts, whereas `extract_keyframes` relies on the video encoder's existing I-frame metadata (keyframes) which are accessible without full reconstruction of intermediate frames. This architectural difference means balanced mode performs content analysis while efficient mode performs metadata reading.

### What happens if a video has no scene cuts or very few keyframes?

Both functions implement fallback safeguards: `extract_keyframes` falls back to uniform sampling if fewer than `KEYFRAME_MIN` keyframes are detected, while `extract_scene_or_uniform` falls back to uniform fps-based extraction when fewer than `SCENE_MIN_FRAMES` cuts are found. These fallbacks ensure you receive the requested number of frames even for static content, though the fallback extraction method varies by function.

### How do I adjust the frame extraction limits for custom requirements?

Both functions accept a `max_frames` parameter (defaulting to 50 for efficient and 100 for balanced) and `dedup` boolean for near-duplicate removal. When using the CLI in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py), these defaults are hardcoded based on the `--detail` flag selected, but you can modify the Python API calls directly to set custom caps such as `max_frames=200` for high-detail analysis or reduce to `max_frames=10` for thumbnail generation.