# How Keyframe Extraction Mode Works in bradautomates/claude-video for Efficient Detail

> bradautomates/claude-video keyframe extraction decodes only I-frames for efficient detail. See how it uses ffmpeg, deduplication, and frame budgets for fast, representative frames without heavy decoding.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-07-09

---

**The keyframe extraction mode in bradautomates/claude-video uses ffmpeg to decode only I-frames (keyframes), applies lightweight perceptual deduplication, and enforces a strict frame budget to deliver scene-representative frames without the computational cost of full video decoding.**

The `bradautomates/claude-video` repository provides a video analysis skill that leverages multiple extraction strategies to balance speed and detail. Among these, the **keyframe extraction mode** serves as the fastest "detail-engine" in the `watch` skill, designed to capture distinct visual moments while maintaining predictable processing costs. This mode prioritizes efficiency by exploiting the video encoder's existing scene-cut decisions rather than processing every frame.

## Fast Keyframe Decoding with ffmpeg

The extraction pipeline begins in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) where the `extract_keyframes` function constructs an ffmpeg command optimized for speed. By invoking ffmpeg with the `-skip_frame nokey` flag (lines 1002-1018), the decoder reconstructs **only the keyframes** that the encoder originally emitted at scene cuts, skipping all P-frames and B-frames. This avoids the CPU overhead of full video decoding while naturally aligning extracted frames with significant visual transitions.

## Timestamp Extraction and Candidate Building

As ffmpeg processes the keyframes, the `showinfo` filter outputs `pts_time` entries for each decoded frame. The code uses the `SHOWINFO_TS_RE` regex (defined at line 39) to parse these timestamps in the output loop (lines 1024-1030), pairing each extracted frame with its exact source time. Between lines 1027 and 1034, the function constructs candidate dictionaries containing the frame index, timestamp in seconds, file path, and a reason field set to `"keyframe"`.

## Minimum Keyframe Guard and Fallback Logic

To handle static content that might lack sufficient scene changes, the implementation enforces a `KEYFRAME_MIN` threshold of **4 keyframes**. If the initial extraction yields fewer than four candidates (lines 1036-1050), the engine automatically falls back to uniform extraction mode. This ensures users always receive a usable frame set even when processing talking-head videos or static recordings with minimal encoder keyframe emissions.

## Perceptual Deduplication Pipeline

After building the candidate list, the system applies perceptual deduplication via the `dedupe_perceptual` function (lines 64-71). The pipeline generates **16×16 pixel grayscale thumbnails** for each candidate and calculates mean-pixel differences between them. When the difference falls at or below the `DEDUP_THRESHOLD` of **2.0**, frames are considered near-identical and deleted. This removes redundant shots while preserving visual variety without the computational cost of sophisticated perceptual hashing algorithms.

## Even Sampling to Frame Budget

Finally, the `_even_sample` helper function (lines 93-100) evenly distributes the remaining frames across the video timeline to respect the user-specified `max_frames` budget. This sampling strategy always retains the first and last frames while selecting intermediate frames at regular intervals, guaranteeing temporal coverage of the entire clip without exceeding the frame limit. The function returns the selected frames together with metadata describing the engine used, candidate counts, deduplicated frames, and whether a fallback occurred (lines 1069-1082).

## Usage Examples

### Python API

You can call `extract_keyframes` directly from your Python code to process videos programmatically:

```python
from pathlib import Path
from skills.watch.scripts.frames import extract_keyframes

video_path = "example.mp4"
output_dir = Path("frames_out")

# Extract up to 50 keyframes at 512px resolution with deduplication enabled

frames, meta = extract_keyframes(
    video_path,
    output_dir,
    resolution=512,
    max_frames=50,
    start_seconds=None,
    end_seconds=None,
    dedup=True,
)

print("Engine used:", meta["engine"])
print("Chosen frames:", len(frames))

# Each entry contains: index, timestamp_seconds, path, reason

```

### Command Line Interface

The script also exposes a CLI for shell-based workflows:

```bash
python -m skills.watch.scripts.frames \
    example.mp4 frames_out \
    --resolution 768 \
    --max-frames 30 \
    --no-dedup

```

This emits a JSON summary on stdout identical to the Python API return value.

### Integration in the Watch Workflow

The high-level [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) orchestrator invokes this mode when users select the **"keyframe"** detail setting:

```python

# Inside skills/watch/scripts/watch.py

if detail_mode == "keyframe":
    frames, meta = extract_keyframes(
        video, out_dir, max_frames=budget, dedup=True
    )

```

The extracted frames are then merged with transcript-cue frames and sent to the LLM for analysis.

## Summary

- **Keyframe-only decoding** uses `ffmpeg -skip_frame nokey` to process only I-frames, drastically reducing CPU load compared to full video decoding.
- **Automatic fallback** to uniform extraction occurs when fewer than 4 keyframes are detected, ensuring usability across all video types.
- **Perceptual deduplication** removes near-duplicate frames using lightweight 16×16 grayscale thumbnails and a threshold of 2.0 mean-pixel difference.
- **Even sampling** distributes frames across the timeline to respect `max_frames` budgets while preserving first and last frame coverage.
- **Metadata tracking** returns detailed statistics about the extraction process, including candidate counts and fallback status.

## Frequently Asked Questions

### What makes keyframe extraction faster than uniform sampling?

Keyframe extraction skips all predictive and bidirectional frames, decoding only the keyframes that the encoder already flagged as scene changes. According to the source code in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), this reduces the decoding workload to a fraction of the total frame count while still capturing the most visually distinct moments in the video.

### When does the engine fall back to uniform extraction?

The engine falls back when it detects fewer than `KEYFRAME_MIN` (4) keyframes in the source video, as implemented in lines 1036-1050 of [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py). This typically occurs with static content like screen recordings or talking-head videos where the encoder emits minimal keyframes.

### How does the deduplication threshold work?

The `DEDUP_THRESHOLD` constant (set to 2.0) defines the maximum mean-pixel difference between 16×16 grayscale thumbnails before two frames are considered duplicates. If the difference is ≤ 2.0, the `dedupe_perceptual` function (lines 64-71) deletes the redundant frame, ensuring the final set contains only visually distinct images.

### Can I disable deduplication for keyframe mode?

Yes. Both the Python API and CLI support disabling deduplication by setting `dedup=False` or passing the `--no-dedup` flag, respectively. This retains all extracted keyframes without the perceptual comparison step, useful when you need maximum frame density for analysis.