# Uniform Fallback for Frame Extraction in Claude Video: Implementation Guide

> Implement uniform fallback frame extraction in Claude Video to ensure usable frame sequences for static content like screen recordings. Get our implementation guide.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: how-to-guide
- Published: 2026-07-29

---

**Claude Video automatically falls back to uniform frame sampling when scene detection identifies fewer than 8 distinct shots, ensuring static content like screen recordings still generate usable frame sequences for analysis.**

The `bradautomates/claude-video` repository implements an intelligent two-stage extraction engine that balances quality with reliability. When processing videos with minimal visual changes, the **uniform fallback for frame extraction** guarantees consistent output by switching from scene-based to time-based sampling. This mechanism is critical for handling screen recordings, talking-head videos, and other low-motion content that would otherwise yield insufficient frames for analysis.

## How the Uniform Fallback Works

The extraction logic in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) operates through a conditional pipeline that prioritizes scene detection before resorting to uniform sampling.

### Scene Detection Stage

First, the system runs ffmpeg's scene-change detector via `extract_scene_candidates` to identify visual cuts exceeding `SCENE_THRESHOLD`. This captures the first frame and every subsequent frame where significant visual differences occur, creating a candidate list of scene boundaries.

### The Fallback Threshold

The system compares the detected scene count against `SCENE_MIN_FRAMES` (default 8). If the video contains sufficient visual changes, the engine retains the scene-based frames, optionally applying perceptual deduplication and even-sampling to meet the user's `max_frames` limit.

### Uniform Sampling Activation

When fewer than 8 scenes are detected, the code calculates a `fallback_cap` budget and invokes the `extract` function. This uses ffmpeg's `fps=` filter to grab frames at constant intervals, ensuring coverage across the entire video duration regardless of visual staticity. After extraction, `dedupe_perceptual` removes near-duplicate frames if the `dedup` parameter is enabled.

## Implementation in frames.py

The core logic resides in the `extract_scene_or_uniform` function within [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py):

```python
def extract_scene_or_uniform(
    video_path: str,
    out_dir: Path,
    fps: float,
    target_frames: int,
    resolution: int = 512,
    max_frames: int | None = 100,
    start_seconds: float | None = None,
    end_seconds: float | None = None,
    dedup: bool = True,
) -> tuple[list[dict], dict]:
    """Prefer scene selection, falling back to uniform only when the video is
    effectively static (fewer than ``SCENE_MIN_FRAMES`` detected shots)."""
    # 1️⃣ Run full-range scene detection (uncapped)

    scene_frames = extract_scene_candidates(
        video_path,
        out_dir,
        resolution=resolution,
        max_frames=None,
        start_seconds=start_seconds,
        end_seconds=end_seconds,
    )
    scene_count = len(scene_frames)

    # 2️⃣ If enough cuts → keep them (optionally dedupe & even-sample)

    if scene_count >= SCENE_MIN_FRAMES:
        # ... dedup, cap, and return selected frames

        pass

    # 3️⃣ Otherwise fall back to uniform extraction

    fallback_cap = target_frames if max_frames is None else min(max_frames, target_frames)
    frames = extract(
        video_path,
        out_dir,
        fps=fps,
        resolution=resolution,
        max_frames=fallback_cap,
        start_seconds=start_seconds,
        end_seconds=end_seconds,
    )
    if dedup:
        frames, n_dropped = dedupe_perceptual(frames)
    return frames, {
        "engine": "uniform",
        "candidate_count": scene_count,
        "deduped_count": n_dropped,
        "selected_count": len(frames),
        "fallback": True,
    }

```

## Using the Fallback via CLI and API

### CLI Usage with watch.py

The top-level entry point [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) automatically invokes the fallback when using `--detail balanced` or `--detail token-burner` modes:

```bash
watch "https://example.com/static-screen-recording.mp4" \
      --detail balanced \
      --resolution 640 \
      --max-frames 30

```

The generated markdown report indicates fallback activation with metadata showing `"engine": "uniform"` and `"fallback": true`, along with the final frame count and deduplication statistics.

### Direct API Integration

To programmatically leverage the fallback mechanism:

```python
from pathlib import Path
from skills.watch.scripts.frames import (
    get_metadata,
    auto_fps,
    extract_scene_or_uniform,
)

video = "lecture.mp4"
out_dir = Path("frames")
metadata = get_metadata(video)

# Compute FPS for 50 frames across 10 seconds

fps, _ = auto_fps(duration_seconds=10, max_frames=50)

frames, meta = extract_scene_or_uniform(
    video_path=video,
    out_dir=out_dir,
    fps=fps,
    target_frames=50,
    resolution=512,
    max_frames=50,
)

print("Engine:", meta["engine"])          # "scene" or "uniform"

print("Fallback:", meta["fallback"])      # True only if uniform path used

```

## Configuration Parameters

The fallback behavior is controlled by constants defined in the codebase:

- **SCENE_MIN_FRAMES**: Default threshold of 8 scenes required to avoid fallback
- **SCENE_THRESHOLD**: Visual difference sensitivity for scene detection
- **dedup**: Boolean flag (default `True`) enabling perceptual deduplication via `dedupe_perceptual` after uniform extraction

Configuration defaults and detail-level mappings reside in [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py), which [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) references when determining extraction parameters.

## Summary

- Claude Video implements a **uniform fallback for frame extraction** that activates when scene detection finds fewer than `SCENE_MIN_FRAMES` (8) distinct shots.
- The logic is centralized in `extract_scene_or_uniform` within [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py).
- Fallback extraction uses ffmpeg's constant FPS filtering via the `extract` function to ensure comprehensive temporal coverage.
- The system automatically applies perceptual deduplication to remove near-duplicate frames from uniform samples.
- CLI users trigger this behavior through [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) with `--detail balanced`, while API users can call `extract_scene_or_uniform` directly.

## Frequently Asked Questions

### What triggers the uniform fallback in Claude Video?

The fallback triggers when `extract_scene_candidates` returns fewer than `SCENE_MIN_FRAMES` (default 8) distinct scenes. This typically occurs with static content like screen recordings or talking-head videos where visual changes fall below the `SCENE_THRESHOLD` sensitivity, causing the scene detector to identify insufficient cut points.

### How does the fallback handle frame deduplication?

After uniform extraction, the system runs `dedupe_perceptual` to remove near-duplicate frames when the `dedup` parameter is `True` (the default). The metadata dictionary returned by `extract_scene_or_uniform` includes `deduped_count`, indicating exactly how many frames were removed from the initial uniform sample set.

### Can I force uniform extraction without scene detection?

Yes. Bypass `extract_scene_or_uniform` and call `extract` directly from [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py). This function uses ffmpeg's `fps=` filter for constant-rate sampling without scene analysis, though you will lose the automatic fallback intelligence and the unified metadata reporting that indicates which engine was used.

### Where is the fallback logic implemented in the codebase?

The primary implementation lives in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) inside the `extract_scene_or_uniform` function. The CLI integration resides in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py), which determines which extraction engine to invoke based on the `--detail` flag and orchestrates the final report generation.