# Uniform Sampling Fallback in Claude-Video: How Scene Detection Gracefully Degrades

> Claude-Video uses uniform sampling fallback when scene detection yields too few cuts. Ensure usable frame sets with this graceful degradation technique.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: internals
- Published: 2026-08-03

---

**When scene detection finds fewer than 4 cuts, the `extract_scene_or_uniform` function automatically switches to uniform time-based sampling to guarantee a usable frame set.**

The **claude-video** repository by bradautomates implements a robust frame extraction pipeline in its `watch` skill. The **uniform sampling fallback** ensures that videos always yield usable frames—even when scene detection fails, as with static screen recordings or short clips. This article explains the fallback mechanism implemented in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), how it respects user frame budgets, and the metadata it returns for downstream processing.

## How Scene Detection and Fallback Work Together

The entry point for frame extraction is `extract_scene_or_uniform`, which orchestrates a two-phase approach: attempt scene detection first, then fall back to uniform sampling if too few cuts are found.

### Phase 1: Scene-Cut Detection

The function calls `extract_scene_candidates` to identify frames corresponding to visual scene changes. This produces a list of candidate timestamps where the video content shifts significantly.

### Phase 2: The Minimum-Cut Check

The detected cut count is compared against `SCENE_MIN_FRAMES`, a constant set to **4** according to [`SKILL.md`](https://github.com/bradautomates/claude-video/blob/main/SKILL.md). The logic branches based on this threshold:

- **≥ 4 cuts**: Frames are optionally deduplicated, then evenly sampled using the `"scene"` engine
- **< 4 cuts**: The **uniform sampling fallback** triggers automatically

## The Fallback Implementation

When scene detection under-delivers, the code switches to time-based extraction. Here is the exact fallback logic from [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) (lines 21–34):

```python
fallback_cap = target_frames if max_frames is None else min(max_frames, target_frames)
frames = extract(
    video_path,
    out_dir,
    fps=fps,
    resolution=resolution,
    max_frames=fallback_cap,
    start_seconds=start_seconds,
    end_seconds=end_seconds,
)

```

### Key Fallback Behaviors

- **`fallback_cap`** — Respects both the user-requested `target_frames` and any hard `max_frames` limit, taking the minimum of the two when both are specified
- **`extract`** — Performs simple time-based extraction at the supplied `fps`, producing evenly-spaced frames across the clip duration
- **`start_seconds` / `end_seconds`** — Honor trim boundaries even in fallback mode

## Optional Perceptual Deduplication

When `dedup=True`, uniformly sampled frames pass through `dedupe_perceptual` before return. This removes near-identical frames based on perceptual thumbnail difference—useful when uniform sampling hits identical frames in static video regions.

## Metadata Signaling the Fallback

The function returns a metadata dictionary that explicitly logs the fallback event:

```json
{
  "engine": "uniform",
  "candidate_count": 2,
  "deduped_count": 0,
  "selected_count": 30,
  "fallback": true
}

```

| Field | Meaning |
|-------|---------|
| `engine` | `"uniform"` or `"scene"` — indicates which engine produced frames |
| `candidate_count` | Scene cuts detected before fallback decision |
| `deduped_count` | Frames removed by perceptual deduplication |
| `selected_count` | Final frame count returned |
| `fallback` | Boolean flag, `true` only when uniform fallback activated |

## Practical Code Examples

### Example 1: Fallback Triggered on Static Video

```python
from pathlib import Path
from skills.watch.scripts.frames import extract_scene_or_uniform

# 30-second static screen recording with minimal visual changes

frames, meta = extract_scene_or_uniform(
    video_path="static_screen.mp4",
    out_dir=Path("frames"),
    fps=1.0,
    target_frames=100,
    resolution=512,
    max_frames=100,
    dedup=True,
)

print(meta)

# {

#   "engine": "uniform",

#   "candidate_count": 2,      # only 2 scene cuts detected

#   "deduped_count": 0,

#   "selected_count": 30,      # 30 seconds × 1 fps

#   "fallback": True,

# }

```

### Example 2: Scene Engine Succeeds

```python
frames, meta = extract_scene_or_uniform(
    video_path="movie_clip.mp4",
    out_dir=Path("frames"),
    fps=2.0,
    target_frames=120,
    resolution=512,
    max_frames=100,
    dedup=True,
)

assert meta["engine"] == "scene"
assert meta["fallback"] is False

```

## Where This Logic Lives

| File | Role |
|------|------|
| [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) | Core implementation of `extract_scene_or_uniform` and the uniform fallback |
| [`skills/watch/SKILL.md`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/SKILL.md) | Documents the "fewer than four keyframes" fallback policy |
| [`tests/test_frames.py`](https://github.com/bradautomates/claude-video/blob/main/tests/test_frames.py) | Unit tests asserting `meta["engine"] == "uniform"` and `meta["fallback"] is True` |
| [`README.md`](https://github.com/bradautomates/claude-video/blob/main/README.md) | Mentions fallback behavior in detail-level descriptions |

## Summary

- The **uniform sampling fallback** activates automatically when `extract_scene_candidates` returns fewer than `SCENE_MIN_FRAMES` (4) cuts
- The fallback respects frame budgets through `fallback_cap = min(max_frames, target_frames)` and honors trim boundaries
- Uniform frames can be perceptually deduplicated via `dedupe_perceptual` when `dedup=True`
- Metadata returned includes `engine`, `candidate_count`, `fallback` flag, and final selection counts
- Tests in [`tests/test_frames.py`](https://github.com/bradautomates/claude-video/blob/main/tests/test_frames.py) verify fallback behavior for downstream consumers

## Frequently Asked Questions

### What triggers the uniform sampling fallback in claude-video?

The fallback triggers when scene detection finds fewer than 4 candidate cuts. This constant `SCENE_MIN_FRAMES` is checked in `extract_scene_or_uniform` within [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py). Videos with minimal motion—static screen recordings, slideshows, or very short clips—commonly hit this threshold.

### Does the fallback respect my frame budget and maximum limits?

Yes. The `fallback_cap` calculation takes `min(max_frames, target_frames)` when both are provided, ensuring the uniform sampler never exceeds your hard limit while attempting to reach your target. If only `target_frames` is specified, that becomes the cap.

### How can I detect from code that the fallback occurred?

Check `meta["fallback"]` in the return value from `extract_scene_or_uniform`. This boolean is `True` only when uniform sampling was activated. The `meta["engine"]` field will also read `"uniform"` instead of `"scene"`.

### Is deduplication applied to uniformly sampled frames?

Yes, when `dedup=True`. The uniformly sampled frames are passed through `dedupe_perceptual`, which removes perceptually similar frames. The `meta["deduped_count"]` field reports how many duplicates were removed from the final set.