# How the Uniform Sampler Fallback Mechanism Works in claude-video

> Understand the uniform sampler fallback mechanism in claude-video. Learn how it ensures usable image sets when scene detection fails by switching to even temporal sampling.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: internals
- Published: 2026-07-25

---

**The uniform sampler fallback mechanism automatically activates when the scene detection or keyframe extraction engines fail to generate enough distinct frames, switching to evenly-spaced temporal sampling to guarantee a usable set of images.**

In the `bradautomates/claude-video` repository, the uniform sampler serves as the critical safety net that ensures consistent frame extraction across diverse video types. When higher-level analysis engines cannot identify sufficient visual changes, this **fallback mechanism** seamlessly transitions to uniform temporal sampling to meet the requested frame budget.

## When the Fallback Mechanism Triggers

The uniform sampler activates in two specific scenarios within [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py):

### Scene Detection Engine

When the scene-cut detector returns fewer than **8** distinct shots (`SCENE_MIN_FRAMES`), the video is treated as either static or insufficiently varied for scene-based analysis. The `extract_scene_or_uniform()` function (lines 54‑55) detects this condition and falls back to `extract()`, sampling frames at a constant rate calculated by `auto_fps()` for the effective duration of the clip.

### Keyframe Extraction Engine

If fewer than **4** keyframes (`KEYFRAME_MIN`) are discovered—typically occurring in very short clips or strangely encoded files—the `extract_keyframes()` function (lines 636‑638) discards the sparse candidates and rebuilds the sample set using the same uniform extraction method.

## The Fallback Execution Process

The uniform sampler follows a consistent four-step procedure to ensure reliable output:

1. **Detect insufficient candidates**
   
   ```python
   if scene_count < SCENE_MIN_FRAMES:
       # Trigger fallback in extract_scene_or_uniform()

   if len(candidates) < KEYFRAME_MIN:
       # Trigger fallback in extract_keyframes()

   ```

2. **Calculate optimal frame rate**
   
   ```python
   fps, _ = auto_fps(eff_duration, max_frames=budget)
   ```

   
   The `auto_fps()` function (lines 22‑36) computes a frame rate that respects the `max_frames` budget while distributing samples evenly across the video's effective duration.

3. **Execute uniform extraction**
   
   ```python
   frames = extract(
       video_path, out_dir, fps=fps,
       resolution=resolution, max_frames=budget,
       start_seconds=start_seconds, end_seconds=end_seconds,
   )
   ```

   
   The `extract()` function (lines 71‑84) invokes `ffmpeg` with the `fps={fps}` filter to generate evenly-spaced JPEGs.

4. **Apply optional deduplication**
   
   ```python
   if dedup:
       frames, n_dropped = dedupe_perceptual(frames)
   ```

   
   The `dedupe_perceptual()` function (lines 64‑71) removes visually identical frames that may result from static video segments.

## Metadata and Engine Identification

Regardless of which engine initially attempted extraction, the fallback mechanism consistently marks the operation in the returned metadata dictionary (lines 68‑73 and 662‑668):

```python
meta = {
    "engine": "uniform",
    "fallback": True,
    "frame_count": len(frames)
}

```

This allows downstream components in the `/watch` skill to report that uniform sampling was used instead of the primary extraction method.

## Code Examples

### Scene Engine with Fallback

```python
from skills.watch.scripts.frames import extract_scene_or_uniform
from pathlib import Path

frames, meta = extract_scene_or_uniform(
    video_path="example.mp4",
    out_dir=Path("./out"),
    target_frames=30,
    max_frames=30,
)

print(meta["engine"])    # "scene" or "uniform"

print(meta["fallback"])  # False for scene, True for uniform fallback

```

### Keyframe Extraction with Automatic Fallback

```python
from skills.watch.scripts.frames import extract_keyframes
from pathlib import Path

frames, meta = extract_keyframes(
    video_path="short_clip.mp4",
    out_dir=Path("./out_kf"),
    max_frames=10,
)

print(meta["engine"])    # "keyframe" or "uniform"

print(meta["fallback"])  # True when keyframe count < 4

```

## Summary

- The **uniform sampler fallback mechanism** activates when scene detection yields fewer than 8 frames or keyframe extraction finds fewer than 4 candidates.
- The system calculates an optimal frame rate using `auto_fps()` that respects the `max_frames` budget while ensuring even temporal distribution.
- The `extract()` function in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) uses `ffmpeg` with a constant FPS filter to generate the fallback frame set.
- Optional **perceptual deduplication** removes visually identical frames after extraction.
- Metadata flags (`engine="uniform"`, `fallback=True`) ensure downstream processes can identify when the fallback was invoked.

## Frequently Asked Questions

### What triggers the uniform sampler fallback in claude-video?

The fallback triggers when the primary extraction engines cannot produce enough distinct frames. Specifically, when the scene-cut detector finds fewer than 8 shots (`SCENE_MIN_FRAMES`) or when keyframe extraction yields fewer than 4 candidates (`KEYFRAME_MIN`). In both cases, the system automatically switches to uniform temporal sampling to guarantee a usable frame set.

### How does the uniform sampler calculate how many frames to extract?

The `auto_fps()` function (lines 22‑36 in [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py)) calculates a frame rate that distributes the requested `max_frames` budget evenly across the video's effective duration. This ensures the output contains the optimal number of evenly-spaced frames without exceeding the user's specified limits.

### Can I prevent the uniform sampler from running if the video is static?

No, the fallback is automatic and designed as a safety mechanism. However, you can apply the optional `dedupe_perceptual()` filter after extraction, which removes visually identical frames that often result from static video content. This function is available at lines 64‑71 in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py).

### Where is the uniform sampler fallback implemented in the codebase?

The core implementation resides in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) in the `bradautomates/claude-video` repository. The `extract_scene_or_uniform()` function handles the scene detection fallback, while `extract_keyframes()` manages the keyframe extraction fallback. Both functions call the `extract()` function (lines 71‑84) to perform the actual uniform sampling using `ffmpeg`.