# Claude-Video FFmpeg Command Flow for Scene Cut and Keyframe Extraction

> Discover the FFmpeg command flow Claude-Video uses for scene cut and keyframe extraction. Learn how to decode only salient frames using specific filters and flags.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: internals
- Published: 2026-07-14

---

**Claude-Video utilizes two distinct FFmpeg pipelines—one leveraging the `select='gt(scene,THRESH)'` video filter for scene-change detection and another employing the `-skip_frame nokey` flag for I-frame extraction—to decode only salient frames while harvesting timestamps from `showinfo` stderr output.**

The `claude-video` repository by bradautomates implements intelligent frame sampling through targeted FFmpeg commands defined in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py). Rather than uniformly extracting frames at fixed intervals, the codebase selectively decodes visually significant moments using FFmpeg’s built-in scene detection and keyframe indexing capabilities.

## Scene-Cut Frame Extraction Pipeline

The scene-cut extraction pipeline identifies abrupt visual transitions to capture representative frames from distinct shots.

### FFmpeg Command Construction

In [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), the `extract_scene_candidates` function (lines 17-41, 44-63) constructs an FFmpeg command utilizing the **scene detection filter**:

```bash
ffmpeg -hide_banner -loglevel info -y \
  -ss START -to END \
  -i INPUT_VIDEO \
  -vf "select='eq(n\,0)+gt(scene\,THRESH)',scale=WIDTH:HEIGHT:force_original_aspect_ratio=decrease:force_divisible_by=2,showinfo" \
  -vsync vfr \
  -q:v 4 \
  OUTPUT_PATTERN

```

**Key parameters:**
- `select='eq(n\,0)+gt(scene\,THRESH)'` – Retains the first frame (`eq(n,0)`) and any frame where the scene-change confidence exceeds the threshold (default **0.20**).
- `showinfo` – Emits frame metadata to stderr, including presentation timestamps.
- `-vsync vfr` – Outputs only the selected frames as individual JPEGs without duplicating frames to maintain constant frame rate.

### Timestamp Harvesting

After execution, the code parses the stderr output using the regex pattern `pts_time:([0-9.]+)` to extract precise timestamps for each saved frame. This allows downstream processing to associate specific times with extracted images without re-scanning the video.

## Key-Frame (I-Frame) Extraction Pipeline

For videos where scene detection may fail or for alternative sampling strategies, Claude-Video provides a **keyframe-only extraction** method that decodes exclusively I-frames produced by the video encoder.

### FFmpeg Command Structure

The `extract_keyframes` function (lines 75-85, 100-124) implements this pipeline:

```bash
ffmpeg -hide_banner -loglevel info -y \
  -skip_frame nokey \
  -ss START -to END \
  -i INPUT_VIDEO \
  -vf "scale=WIDTH:HEIGHT:force_original_aspect_ratio=decrease:force_divisible_by=2,showinfo" \
  -vsync vfr \
  -q:v 4 \
  OUTPUT_PATTERN

```

**Critical flags:**
- `-skip_frame nokey` – Instructs the decoder to process **only keyframes** (I-frames), skipping all predictive frames. This significantly reduces CPU load while capturing structurally complete images that typically align with scene boundaries.
- Identical scaling and timestamp extraction logic as the scene-cut pipeline.

### Fallback Mechanism

If the encoder generated fewer than **4 keyframes** (configurable via `SCENE_MIN_FRAMES` at line 22-24), the function automatically falls back to uniform FPS-based extraction to ensure sufficient frame coverage.

## Implementation Architecture

The frame extraction workflow follows a structured pipeline defined across several functions in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py):

1. **Metadata Gathering** – `get_metadata` runs `ffprobe` (lines 86-20) to determine duration, dimensions, and codec information before selecting extraction parameters.

2. **Budget Calculation** – `auto_fps` or `auto_fps_focus` (lines 22-39) computes an appropriate frame rate or budget based on video length and user-specified `max_frames`.

3. **Extraction Dispatch** – `extract_scene_or_uniform` decides whether to return scene-cut candidates or fall back to uniform extraction based on the minimum frame threshold.

4. **Deduplication** – `dedupe_perceptual` (lines 63-78) removes near-identical frames using tiny grayscale thumbnails to prevent redundant processing of visually similar images.

5. **Even Sampling** – `_even_sample` (lines 93-103) evenly distributes the final frame selection across the video duration while preserving the first and last frames, respecting the `max_frames` constraint.

## Usage Examples

### Extract Scene-Cut Frames (Python API)

```python
from pathlib import Path
from skills.watch.scripts.frames import extract_scene_candidates

candidates = extract_scene_candidates(
    video_path="example.mp4",
    out_dir=Path("frames"),
    resolution=512,
    max_frames=100,
    threshold=0.20,
    start_seconds=10,
    end_seconds=60
)

# Returns list of dicts with keys: index, timestamp_seconds, path, reason

```

### Extract Keyframes (Python API)

```python
from pathlib import Path
from skills.watch.scripts.frames import extract_keyframes

frames, meta = extract_keyframes(
    video_path="example.mp4",
    out_dir=Path("keyframes"),
    resolution=512,
    max_frames=50,
    dedup=True
)

print(meta)  # engine='keyframe', candidate_count=..., selected_count=...

```

### Command-Line Interface

```bash
python3 skills/watch/scripts/frames.py example.mp4 frames_out \
    --resolution 640 \
    --max-frames 80

```

This CLI entry point automatically selects the appropriate extraction strategy based on video characteristics and performs post-processing deduplication.

## Summary

- **Scene-cut extraction** uses FFmpeg’s `select='gt(scene,0.20)'` filter combined with `showinfo` to capture frames at visual discontinuities, implemented in `extract_scene_candidates`.
- **Keyframe extraction** leverages `-skip_frame nokey` to decode only I-frames, falling back to uniform sampling if fewer than 4 keyframes exist, implemented in `extract_keyframes`.
- Both pipelines parse `pts_time` values from `showinfo` stderr output to maintain precise temporal metadata.
- Post-processing includes perceptual deduplication (`dedupe_perceptual`) and even sampling (`_even_sample`) to respect frame budgets while maximizing visual coverage.
- All logic resides in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), orchestrated by [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) for the `/watch` skill interface.

## Frequently Asked Questions

### What is the default scene detection threshold in Claude-Video?

The default threshold is **0.20**, defined as the `threshold` parameter in `extract_scene_candidates`. Values closer to 0.0 increase sensitivity (detecting subtle changes), while values approaching 1.0 require drastic scene changes to trigger frame selection.

### How does the pipeline handle videos with insufficient keyframes?

When `-skip_frame nokey` yields fewer than 4 frames (configurable via `SCENE_MIN_FRAMES`), the `extract_keyframes` function automatically falls back to uniform FPS-based extraction. This ensures the system always returns a usable set of frames even for videos with aggressive keyframe optimization or very short durations.

### Why does Claude-Video use `showinfo` instead of `ffprobe` for timestamps?

The `showinfo` video filter emits timestamps to stderr **during** the frame extraction process, eliminating the need for a separate metadata scan. This single-pass approach improves performance by decoding the video once while simultaneously capturing both the JPEG output and temporal data via regex parsing of `pts_time` values.

### Where is the frame extraction logic implemented in the repository?

All FFmpeg command construction and execution logic resides in **[`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py)**. High-level orchestration—including the decision between scene-cut, keyframe, or uniform extraction—occurs in **[`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py)**, while test coverage exists in [`tests/test_frames.py`](https://github.com/bradautomates/claude-video/blob/main/tests/test_frames.py).