# Claude-Video Keyframe Extraction Method: How It Works and When Uniform Sampling Kicks In

> Discover Claude-Video keyframe extraction. Learn how it uses FFmpeg for efficient decoding and when uniform sampling is applied for optimal results.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-08-04

---

**Claude-Video extracts keyframes using FFmpeg's `-skip_frame nokey` filter to decode only I-frames, but falls back to uniform sampling when fewer than 4 keyframes are found in a video segment.**

The `bradautomates/claude-video` repository implements a hybrid frame extraction strategy that prioritizes keyframes for efficiency, with an automatic fallback to uniform temporal sampling when keyframe density is insufficient. This article breaks down the implementation in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) and explains the exact conditions triggering each mode.

## Keyframe Extraction Method in Claude-Video

The core keyframe extraction logic lives in the `extract_keyframes` function found at [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) (lines 776-814). This function implements a sophisticated two-phase approach optimized for video analysis workloads.

### Phase 1: Keyframe-Only Decode with FFmpeg

Claude-Video leverages FFmpeg's frame-skipping capabilities to avoid decoding non-keyframes entirely. The command construction uses `-skip_frame nokey`, which instructs the decoder to discard all frames except I-frames (keyframes):

```python
cmd = [
    "ffmpeg", "-hide_banner", "-loglevel", "info", "-y",
    "-skip_frame", "nokey",
    "-i", str(Path(video_path).resolve()),
    "-vf", f"{_scale_filter(resolution)},showinfo",
    "-vsync", "vfr", "-q:v", "4", output_pattern,
]

```

This approach provides significant performance advantages:

- **Reduced decode overhead**: Only keyframes are processed, skipping P-frames and B-frames
- **Lower memory footprint**: Fewer frames enter the processing pipeline
- **Deterministic output**: Keyframe positions are encoding-dependent but reproducible

The `showinfo` filter in the video filter (`-vf`) chain outputs per-frame metadata to stderr, which the Python code parses to extract frame timestamps (lines 825-834).

### Timestamp Collection and Candidate Building

After FFmpeg execution, the function parses the `showinfo` output to build a list of candidate frames. Each candidate receives metadata marking its extraction reason:

```python

# Conceptual representation of candidate structure

candidate = {
    "path": frame_path,
    "timestamp": parsed_timestamp,
    "reason": "keyframe"  # Always "keyframe" in this phase

}

```

## When Claude-Video Uses Uniform Sampling

The keyframe extraction method includes a **mandatory minimum threshold** that triggers automatic fallback behavior. This is the critical decision point in the extraction pipeline.

### The `KEYFRAME_MIN` Threshold

At lines 836-868, the function evaluates whether the keyframe-only extraction yielded sufficient results:

| Condition | Action |
|-----------|--------|
| `len(candidates) >= KEYFRAME_MIN` (4) | Proceed with keyframe set |
| `len(candidates) < KEYFRAME_MIN` | Discard candidates, trigger uniform fallback |

The constant `KEYFRAME_MIN = 4` represents a hardcoded minimum viable keyframe count. When this threshold is unmet, the function assumes the video segment lacks adequate keyframe density for meaningful analysis.

### Uniform Sampling Fallback Implementation

If fallback triggers, `extract_keyframes` performs these steps (lines 836-868):

1. **Cleanup**: Deletes all partially-extracted keyframe candidates
2. **Duration calculation**: Computes the time range from `start_seconds` to `end_seconds`
3. **Automatic FPS calculation**: Calls `auto_fps()` to determine an even sampling rate based on duration and `max_frames`
4. **Uniform extraction**: Invokes the standard `extract()` function with calculated parameters

The `auto_fps` helper calculates sampling frequency to distribute frames evenly across the video duration while respecting the `max_frames` cap.

## Complete Extraction Workflow

Understanding the full pipeline clarifies how keyframe and uniform methods integrate:

```

┌─────────────────┐     ┌─────────────────────┐     ┌─────────────────┐
│  extract_       │────▶│  FFmpeg -skip_frame │────▶│  Parse showinfo │
│  keyframes()    │     │  nokey              │     │  timestamps     │
└─────────────────┘     └─────────────────────┘     └────────┬────────┘
                                                             │
                              ┌────────────────────────────┘
                              ▼
                    ┌─────────────────┐
                    │  Candidates >=4? │
                    └────────┬────────┘
                             │
              ┌──────────────┼──────────────┐
              ▼ YES                         ▼ NO (uniform sampling)
    ┌─────────────────┐            ┌─────────────────┐
    │  Deduplication  │            │  Delete partial │
    │  (optional)     │            │  keyframes      │
    └────────┬────────┘            └────────┬────────┘
             │                              │
             │                    ┌─────────┴─────────┐
             │                    │  Calculate        │
             │                    │  auto_fps from    │
             │                    │  duration         │
             │                    └─────────┬─────────┘
             │                              │
             │                    ┌─────────▼─────────┐
             │                    │  extract() with   │
             │                    │  uniform FPS      │
             │                    └─────────┬─────────┘
             │                              │
             └──────────────┬───────────────┘
                            ▼
                   ┌─────────────────┐
                   │  Even-sampling  │
                   │  to max_frames  │
                   └─────────────────┘

```

## Practical Code Examples

### Standard Keyframe Extraction with Fallback

This call attempts keyframe extraction and automatically falls back to uniform sampling if needed:

```python
from pathlib import Path
from skills.watch.scripts import frames

video_path = "example.mp4"
out_dir = Path("frames_out")

frames_out, meta = frames.extract_keyframes(
    video_path,
    out_dir,
    resolution=512,      # Max dimension for scaling

    max_frames=50,       # Hard cap on returned frames

    start_seconds=10.0,  # Optional: segment start

    end_seconds=40.0,    # Optional: segment end

    dedup=True,          # Enable perceptual deduplication

)

print(meta)

# Output indicates which engine succeeded: "keyframe" or "uniform"

```

### Manual Uniform Sampling

To bypass keyframe extraction entirely, use the `extract` function directly with calculated FPS:

```python

# For a 30-second segment, sample 30 frames (1 fps)

fps, _ = frames.auto_fps(duration_seconds=30, max_frames=30)

uniform_frames = frames.extract(
    video_path,
    out_dir,
    fps=fps,
    resolution=512,
    max_frames=30,
)

```

## Performance and Quality Considerations

| Aspect | Keyframe Mode | Uniform Mode |
|--------|-------------|--------------|
| **Speed** | Faster (decodes ~1-5% of frames) | Slower (decodes all sampled frames) |
| **Temporal coverage** | Irregular, encoding-dependent | Perfectly even |
| **Best for** | Long videos, standard encodings | Short clips, synthetic/cg content |
| **Memory** | Lower peak usage | Higher proportional to FPS |

The fallback to uniform sampling addresses edge cases where video encoding uses:

- **All-intra encoding** (already uniform, but may have few keyframes)
- **Very short segments** (< few seconds)
- **Non-standard GOP structures** (extremely long keyframe intervals)

## Key Source Files in Claude-Video

- **[`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py)**: Core implementation of `extract_keyframes`, uniform fallback, `auto_fps`, and deduplication
- **[`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py)**: Orchestrates the watch skill, calls `extract_keyframes` with user parameters
- **[`tests/test_frames.py`](https://github.com/bradautomates/claude-video/blob/main/tests/test_frames.py)**: Validates keyframe extraction and fallback behavior
- **[`tests/test_dedup.py`](https://github.com/bradautomates/claude-video/blob/main/tests/test_dedup.py)**: Tests perceptual duplicate removal applied post-extraction

## Summary

- Claude-Video's keyframe extraction method uses **FFmpeg's `-skip_frame nokey`** to decode only I-frames, minimizing processing overhead
- **Uniform sampling triggers automatically** when fewer than 4 keyframes (`KEYFRAME_MIN`) are found in the target video segment
- The fallback occurs at lines 836-868 in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), where candidates are discarded and `extract()` is called with calculated FPS
- Both paths ultimately respect `max_frames` through even sampling and support optional perceptual deduplication
- This hybrid approach balances efficiency for typical content with reliability for edge-case encodings

## Frequently Asked Questions

### What is the minimum keyframe threshold in Claude-Video?

The `KEYFRAME_MIN` constant is set to **4 keyframes**. If a video segment yields 3 or fewer keyframes, the extraction automatically switches to uniform sampling mode.

### How does Claude-Video calculate the FPS for uniform fallback?

The `auto_fps` function computes sampling frequency by dividing the video segment duration by the `max_frames` parameter, ensuring even temporal distribution while respecting the frame cap.

### Can I force uniform sampling even if enough keyframes exist?

Yes. Call `frames.extract()` directly with an explicit `fps` parameter instead of using `extract_keyframes()`. This bypasses the keyframe engine entirely.

### Why would a video have fewer than 4 keyframes?

Common causes include: very short clips (under 1-2 seconds), all-intra codecs where every frame is technically a keyframe (confusing the detection), or videos with intentionally long GOP intervals (e.g., 10+ seconds between I-frames).