Understanding the Scene Threshold Constant in Claude-Video Scene Detection

The scene threshold constant (SCENE_THRESHOLD) is a tunable cutoff value defaulting to 0.20 that determines how aggressively ffmpeg detects scene changes when extracting keyframes from video content.

In the bradautomates/claude-video repository, scene-aware frame extraction relies on this precise numerical cutoff to identify meaningful shot boundaries. This scene threshold constant governs the sensitivity of ffmpeg's built-in scene detection algorithm, ensuring the system captures significant visual transitions while filtering out minor fluctuations in the video stream.

What Is the Scene Threshold Constant?

The SCENE_THRESHOLD constant is defined in skills/watch/scripts/frames.py at line 20 with a default value of 0.20. This floating-point value serves as the baseline sensitivity for determining whether a frame represents a substantial scene change or merely a subtle shift between consecutive frames.

How the Scene Threshold Constant Works

Default Parameter in extract_scene_candidates

The constant functions as the default argument for the threshold parameter in the extract_scene_candidates() function (lines 24-25 of skills/watch/scripts/frames.py). When callers invoke this function without specifying a custom threshold, the system automatically applies the 0.20 default.

FFmpeg Filter Construction

Inside extract_scene_candidates(), the threshold value integrates directly into an ffmpeg video filter string constructed at lines 52-53:

vf = f"select='eq(n\\,0)+gt(scene\\,{threshold})',{_scale_filter(resolution)},showinfo"

This filter instructs ffmpeg to select the first frame (eq(n,0)) plus any frame where the internal scene metric exceeds the specified threshold (gt(scene,threshold)). The scene metric represents ffmpeg's calculation of the difference between consecutive frames, normalized to a 0.0-1.0 range.

Frame Selection Logic

The detection process emits two categories of frames:

• First frame: Always captured via eq(n,0) to ensure video coverage starts at the beginning
• Scene-change frames: Captured when the calculated scene difference exceeds the threshold value

The resulting candidate frames include metadata indicating their extraction reason, allowing downstream components to decide whether to retain all detected scenes or apply additional downsampling based on frame budget constraints.

Code Examples for Scene Threshold Usage

Using the Default Threshold

from pathlib import Path
from skills.watch.scripts import frames

video = "example.mp4"
out_dir = Path("tmp/frames")

# Uses SCENE_THRESHOLD = 0.20 internally

scene_frames = frames.extract_scene_candidates(
    video_path=video,
    out_dir=out_dir,
    max_frames=None,          # uncapped – keep every scene change

)
print(scene_frames)           # each dict includes "reason": "scene-change"

Overriding for Aggressive Detection

Lowering the threshold increases sensitivity, detecting more cuts:

# A more aggressive detection (lower threshold → more cuts)

more_cuts = frames.extract_scene_candidates(
    video_path="example.mp4",
    out_dir=Path("tmp/more_cuts"),
    threshold=0.10,          # tighter sensitivity than default 0.20

    max_frames=200,
)

High-Level API Integration

The top-level watch command internally chains through extract_scene_or_uniform, which calls extract_scene_candidates with default parameters:

from skills.watch.scripts import watch

watch.main(["watch", "--detail", "balanced", "example.mp4"])

Verification in Unit Tests

The test suite validates scene engine selection based on threshold detection:

out, meta = frames.extract_scene_or_uniform(
    video_path=str(cut_clip),
    out_dir=tmp_path / "frames",
)
assert meta["engine"] == "scene"           # scene engine used

assert meta["candidate_count"] >= frames.SCENE_MIN_FRAMES

Tuning the Scene Threshold for Different Content

Adjusting the scene threshold constant allows optimization for various video types:

• Lower values (0.10-0.15): Ideal for fast-paced content with rapid cuts, ensuring no significant transitions are missed
• Default value (0.20): Balanced setting suitable for standard dialogue and documentary footage
• Higher values (0.30-0.40): Appropriate for static scenes or surveillance footage where only major lighting changes or camera movements should trigger new keyframes

Summary

• The SCENE_THRESHOLD constant defaults to 0.20 in skills/watch/scripts/frames.py and controls ffmpeg's scene detection sensitivity
• It serves as the default parameter for extract_scene_candidates(), injecting into ffmpeg's select filter to identify frames exceeding the difference cutoff
• Lower thresholds increase detection sensitivity (more frames), while higher thresholds reduce false positives (fewer frames)
• The system always captures the first frame plus any frames where the scene metric exceeds the threshold, creating a candidate pool for downstream processing

Frequently Asked Questions

What is the default value of SCENE_THRESHOLD in Claude-Video?

The default value is 0.20, defined at line 20 of skills/watch/scripts/frames.py. This value provides a balanced sensitivity that detects meaningful shot boundaries without capturing minor frame-to-frame variations caused by compression artifacts or subtle motion.

How does the scene threshold constant affect ffmpeg scene detection?

The constant feeds directly into ffmpeg's select video filter as the comparison value against the scene metric. When ffmpeg calculates that the difference between consecutive frames exceeds this threshold, it flags that frame as a scene change, causing extract_scene_candidates() to emit it as a keyframe candidate with the reason "scene-change".

Can I override the default scene threshold when processing videos?

Yes, the extract_scene_candidates() function accepts an optional threshold parameter that overrides the default constant. You can pass any float value between 0.0 and 1.0 to adjust sensitivity, with lower values detecting more cuts and higher values requiring more substantial visual differences to trigger scene changes.

Where is the scene threshold constant defined in the source code?

The constant is defined in skills/watch/scripts/frames.py at line 20. The same file contains the extract_scene_candidates() function (lines 24-25 for the signature, lines 52-53 for the ffmpeg filter construction) that utilizes this value to construct the scene detection filter string and drive the extraction logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}