# How Claude Video's Efficient Frame Extraction Mode Works: A Deep Dive into Keyframe Sampling

> Learn how Claude Video's efficient frame extraction mode uses ffmpeg keyframe sampling for faster, lower token usage, preserving visual context with deduplication.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-07-28

---

**Claude Video's efficient frame extraction mode limits output to 50 keyframes using ffmpeg's keyframe-only filter, prioritizing speed and low token consumption while maintaining visual context through perceptual deduplication.**

The `bradautomates/claude-video` repository provides a video analysis framework that processes visual content for AI consumption. Its **efficient frame extraction mode** offers the fastest processing path by strategically sampling only essential frames rather than analyzing every scene change or performing uniform sampling across the timeline.

## Selecting the Keyframe Engine in watch.py

The efficient mode diverges from standard processing at the engine selection stage. In [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py), the code evaluates the `detail` parameter to determine which extraction strategy to employ.

When `detail == "efficient"`, the system explicitly selects the keyframes engine instead of the scene-aware alternative:

```python
engine_label = "keyframes" if detail == "efficient" else "scene-aware frames"

```

This logic appears at lines 198–204 of [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py), establishing the processing pipeline that will handle the video. By choosing the keyframes engine, the system signals that it will rely on the video's existing compression keyframes rather than computing scene differences.

## Enforcing the 50-Frame Budget via config.py

Efficient mode operates under a strict resource constraint defined in [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py). The `frame_cap()` function returns a hard limit of 50 frames when the efficient detail level is specified:

```python
if detail == "efficient":  # in config.frame_cap()

    return 50

```

This cap, found at lines 66–67, works in conjunction with `auto_fps_focus` (for targeted segments) or `auto_fps` (for full-video scans) to calculate an appropriate sampling rate. These functions determine the target FPS needed to distribute the frame budget across the video duration without exceeding the 50-frame maximum.

## Extracting Keyframes with FFmpeg in frames.py

The heavy lifting occurs in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) within the `extract_keyframes` function starting at line 576. This implementation leverages **ffmpeg** with the `-skip_frame nokey` flag to output only frames marked as keyframes in the video codec:

```python
def extract_keyframes(...):
    # ffmpeg command selects keyframes only and respects the cap

    # -skip_frame nokey ensures only I-frames are extracted

```

The function accepts a `max_frames` parameter that enforces the budget cap calculated earlier. Once the extraction reaches 50 frames, the process terminates, ensuring compliance with the efficient mode constraints regardless of video length.

## Perceptual Hash Deduplication

By default, the extracted keyframes undergo a deduplication pass using `dedupe_perceptual`. This perceptual hashing algorithm identifies and removes visually identical or near-identical frames that may occur when keyframes cluster in high-motion sequences or static scenes.

The deduplication step ensures that the final frame set contains maximum visual diversity within the 50-frame budget, preventing wasted tokens on redundant imagery while maintaining representative coverage of the video content.

## Merging Transcript Cue Frames

When transcript timestamps accompany the video, efficient mode prioritizes **cue frames** extracted at speech boundaries. The system extracts these cue frames first, then fills the remaining budget with keyframes from the general extraction process.

This hybrid approach ensures that visually important moments aligned with dialogue are preserved even if they don't coincide with compression keyframes. The merge operation does not expand the 50-frame cap; instead, it allocates the budget strategically between transcript-aligned and general visual content.

## Comparing Efficient Mode to Scene-Aware Extraction

Unlike the default scene-aware processing, which analyzes visual differences between frames to detect meaningful transitions, efficient mode relies entirely on the video's existing keyframe structure. **Scene-aware frames** require computationally expensive difference calculations across the entire video, while **efficient keyframe extraction** reads only pre-encoded keyframe markers.

This architectural difference makes efficient mode significantly faster for long videos, though it may miss subtle scene changes that fall between keyframe intervals. The trade-off favors speed and token economy over comprehensive visual analysis.

## Summary

- **Engine Selection**: [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) routes efficient mode to the keyframes engine at lines 198–204, bypassing scene detection algorithms.
- **Hard Cap**: [`config.py`](https://github.com/bradautomates/claude-video/blob/main/config.py) enforces a 50-frame maximum for efficient mode via the `frame_cap()` function.
- **FFmpeg Pipeline**: [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py) uses `-skip_frame nokey` to extract only compression keyframes, respecting the budget via `extract_keyframes`.
- **Deduplication**: Perceptual hashing removes redundant frames to maximize visual diversity within the constrained set.
- **Transcript Integration**: Cue frames from timestamps are prioritized and merged with keyframes without exceeding the 50-frame limit.

## Frequently Asked Questions

### What is the maximum number of frames extracted in efficient mode?

The efficient mode strictly limits extraction to **50 frames** as defined in [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py). This cap applies to the combined total of keyframes and any transcript cue frames, ensuring minimal token usage regardless of video length or complexity.

### Does efficient mode work with video transcripts?

Yes. When transcript timestamps are provided, efficient mode extracts **cue frames** at speech boundaries first, then fills the remaining budget with keyframes. This integration ensures dialogue-aligned visuals are preserved without exceeding the 50-frame cap established in the configuration.

### How does efficient mode differ from scene-aware frame extraction?

Efficient mode relies on **existing compression keyframes** using ffmpeg's `-skip_frame nokey` filter, while scene-aware extraction computes visual differences between frames to detect transitions. According to the source code in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py), scene-aware processing uses a different engine label and typically extracts more frames for comprehensive analysis, whereas efficient mode prioritizes speed and low token cost.

### Why does efficient mode use keyframes instead of uniform sampling?

Keyframes represent natural breakpoints in the video compression where full image data is stored, making them visually significant by definition. Using **keyframes-only extraction** via `extract_keyframes` in [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py) eliminates the computational overhead of analyzing every frame while still capturing structurally important moments. This approach, combined with perceptual deduplication, provides a representative visual summary faster than uniform sampling would allow.