When Does the Claude-Video Scene-Detection Engine Fall Back to Uniform Sampling?

The scene-detection engine in bradautomates/claude-video falls back to uniform sampling whenever it detects fewer than 8 distinct scene-change frames in a video clip.

The bradautomates/claude-video repository provides an intelligent video processing pipeline that automatically selects between scene-based keyframe extraction and uniform frame sampling. Understanding when the scene-detection engine falls back to uniform sampling helps developers optimize video analysis workflows for different content types, from static screen recordings to dynamic cinematic footage.

The 8-Frame Threshold Rule

The scene-detection engine implements a hard threshold to determine whether a video contains enough visual variety to warrant scene-based sampling. According to the source code in skills/watch/scripts/frames.py, the system requires a minimum number of distinct shots to consider the content "scene-rich."

The SCENE_MIN_FRAMES Constant

At line 26 of skills/watch/scripts/frames.py, the threshold is defined as a constant:

SCENE_MIN_FRAMES = 8          # ← https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py#L26

This value represents the minimum number of scene-change frames that must be detected across the entire video clip. If the video contains fewer than 8 distinct scene transitions, the engine assumes the content is effectively static—such as talking-head videos or screen recordings—and switches to uniform sampling to ensure adequate frame coverage.

The Decision Logic

The actual decision occurs inside extract_scene_or_uniform() at lines 542-543:

scene_count = len(scene_frames)
if scene_count >= SCENE_MIN_FRAMES:
    # Proceed with scene-based extraction

else:
    # Trigger uniform sampling fallback

When scene_count falls below 8, the function abandons the scene-detection results and invokes the generic extract() routine, which samples frames at fixed intervals based on the video duration.

Uniform Fallback Implementation

When the fallback triggers, the engine explicitly marks the operation in its metadata return value. At lines 68-73 of skills/watch/scripts/frames.py, the function returns:

return frames, {
    "engine": "uniform",
    "candidate_count": scene_count,
    "deduped_count": n_dropped,
    "selected_count": len(frames),
    "fallback": True,
}

This metadata dictionary indicates that uniform sampling was used ("engine": "uniform") and confirms that a fallback occurred ("fallback": True). The candidate_count field preserves the original number of scenes detected, allowing developers to diagnose why the fallback triggered.

Practical Usage Examples

You can observe this behavior programmatically or via the CLI interface.

Python API Usage

To check whether the fallback occurred when processing a video:

from pathlib import Path
from skills.watch.scripts.frames import extract_scene_or_uniform, auto_fps

video_path = "example.mp4"
out_dir = Path("frames_out")
fps, target = auto_fps(30)               # auto-select an fps budget

frames, meta = extract_scene_or_uniform(
    video_path,
    out_dir,
    fps=fps,
    target_frames=target,
    resolution=512,
    max_frames=100,
)

print(meta["engine"])     # "scene" or "uniform"

print(meta["fallback"])   # True only when uniform fallback occurred

CLI Slash Command

When using the /watch slash-command (implemented in skills/watch/scripts/watch.py at line 215), the fallback happens automatically without user intervention:

/watch https://example.com/video.mp4 "Summarize the video"

The command internally calls extract_scene_or_uniform() and applies the same 8-frame threshold logic, ensuring that static videos still produce useful frame sets for downstream processing.

Summary

• The scene-detection engine requires at least 8 distinct scene-change frames to avoid falling back to uniform sampling.
• The threshold is defined by the SCENE_MIN_FRAMES constant in skills/watch/scripts/frames.py (line 26).
• When fewer than 8 scenes are detected, the extract_scene_or_uniform() function automatically switches to the extract() uniform sampler.
• Fallback events are explicitly marked in the returned metadata with "engine": "uniform" and "fallback": True.
• Both the Python API and the /watch CLI command implement this safeguard to handle static or extremely short videos gracefully.

Frequently Asked Questions

What is the minimum number of scenes required to avoid uniform sampling fallback?

The engine requires 8 distinct scene-change frames as defined by the SCENE_MIN_FRAMES constant at line 26 of skills/watch/scripts/frames.py. Videos with fewer detected scenes are treated as static content and processed using uniform sampling instead.

How can I detect if the uniform sampling fallback was triggered?

Check the metadata dictionary returned by extract_scene_or_uniform(). When the fallback occurs, meta["engine"] equals "uniform" and meta["fallback"] is set to True. The meta["candidate_count"] field also reveals how many scenes were originally detected before the fallback decision.

Where is the scene-detection logic implemented in the claude-video repository?

The core logic resides in skills/watch/scripts/frames.py, specifically within the extract_scene_or_uniform() function (lines 542-543 contain the threshold check). The entry point for CLI usage is skills/watch/scripts/watch.py at line 215, which orchestrates the frame extraction process.

Why does the engine fall back to uniform sampling for static videos?

Static videos—such as screen recordings or single-shot talking heads—produce very few scene-change frames. Without the fallback, the engine might return an insufficient number of keyframes for analysis. Uniform sampling ensures consistent frame coverage across the entire video duration, regardless of visual content changes.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}