What Is the Scene Detection Threshold (0.20) in Claude Video and How Is It Applied?

The scene detection threshold of 0.20 in Claude Video is a minimum visual-difference cutoff that ffmpeg uses to identify distinct shots, defined as SCENE_THRESHOLD in skills/watch/scripts/frames.py and applied through a ffmpeg filter expression that keeps frames only when their scene-change score exceeds this 20% threshold.

Claude Video, an open-source video analysis tool in the bradautomates/claude-video repository, uses this threshold to automatically extract representative frames from video content. Understanding how this value works—and how to adjust it—lets you control exactly how many frames are captured and how sensitive the detection is to subtle visual changes.

Where the 0.20 Threshold Is Defined

The constant originates in the frame extraction module.

In skills/watch/scripts/frames.py at line 20, you'll find the hardcoded default:

SCENE_THRESHOLD = 0.20   # ← source

(view source)

This value serves as the default fallback when callers don't specify their own threshold. The codebase treats it as a sensible middle ground: sensitive enough to catch genuine scene cuts, but conservative enough to avoid flooding output with minor visual fluctuations.

How the Threshold Is Applied in ffmpeg

The actual scene detection happens through ffmpeg's select filter, constructed dynamically in extract_scene_candidates.

At lines 224–226 of skills/watch/scripts/frames.py, the filter string is built as:

select='eq(n\,0)+gt(scene\,{threshold})'

(view source)

This expression combines two conditions with logical OR:

• eq(n\,0) — Always selects the first frame (n == 0), ensuring every video has at least one representative frame regardless of content.
• gt(scene,{threshold}) — Selects any frame where the computed scene-change metric exceeds the threshold (0.20 by default).

The scene metric itself is a per-frame difference score (0.0 to 1.0) that ffmpeg calculates by comparing histograms between consecutive frames. A score of 0.20 therefore represents 20% cumulative visual difference from the previous kept frame.

The Scene Detection Pipeline Step by Step

1. ffmpeg computes scene metrics — Analyzes each frame's color distribution and generates a 0–1 similarity score versus the prior frame.

2. Threshold filtering occurs — Frames with scene ≤ 0.20 are discarded; frames with scene > 0.20 trigger a selection.

3. First frame protection applies — The initial frame is always preserved via eq(n,0), guaranteeing baseline coverage.

4. Candidate list is returned — The extract_scene_candidates function yields timestamps for all selected frames, which downstream processes use for thumbnail generation, deduplication, or analysis.

Practical Code Examples

Using the Default 0.20 Threshold

from pathlib import Path
from skills.watch.scripts import frames

# Rely on the built-in 0.20 threshold

candidates = frames.extract_scene_candidates(
    video_path="example.mp4",
    out_dir=Path("tmp/frames"),
    resolution=512,
    max_frames=100,
)
print(candidates)   # → first frame + frames where scene > 0.20

This demonstrates standard usage—no threshold argument means SCENE_THRESHOLD (0.20) applies automatically.

Adjusting to a More Sensitive Threshold

# Capture more subtle transitions with a lower threshold

candidates = frames.extract_scene_candidates(
    video_path="example.mp4",
    out_dir=Path("tmp/frames"),
    resolution=512,
    max_frames=100,
    threshold=0.10,               # 50% more sensitive than default

)
print(candidates)   # → more frames returned, including minor cuts

Lowering to 0.10 captures dissolves, fades, and rapid motion that 0.20 might ignore. Raising above 0.20 (e.g., 0.30 or 0.40) would restrict output to only dramatic scene changes.

Choosing the Right Threshold Value

Threshold	Behavior	Best For
0.10	High sensitivity; captures minor transitions	Videos with dissolves, fades, or action sequences
0.20 (default)	Balanced; catches clear cuts without excess noise	General-purpose video analysis
0.30+	Conservative; only major scene breaks	Longform content with static shots (interviews, lectures)

The 0.20 default reflects a pragmatic trade-off: according to the bradautomates/claude-video source code, it avoids the over-segmentation common with lower values while still respecting genuine editorial cuts.

Summary

• SCENE_THRESHOLD = 0.20 is defined in skills/watch/scripts/frames.py as the default minimum scene-change score.
• ffmpeg applies this through gt(scene,0.20) in a select filter, alongside mandatory first-frame selection via eq(n,0).
• The threshold represents 20% visual difference; frames below this similarity to their predecessor are skipped.
• Callers can override the default by passing a custom threshold parameter to extract_scene_candidates.

Frequently Asked Questions

What happens if I set the threshold to 0.00?

Setting threshold=0.00 would theoretically select every frame, but in practice ffmpeg's scene metric rarely hits exactly zero for consecutive frames. You'd capture nearly every frame, defeating the purpose of scene detection. Use this only for frame-rate debugging, never production analysis.

Can the threshold be adjusted per-video in Claude Video's CLI?

The /watch skill exposed in skills/watch/SKILL.md doesn't currently surface threshold tuning in its user-facing interface. Advanced users must call extract_scene_candidates directly from Python, as shown in the code examples above, to customize behavior.

Why 0.20 specifically而不是 0.15 or 0.25?

The 0.20 value emerged from empirical testing in bradautomates/claude-video development: it reliably distinguishes intentional scene cuts from minor camera shake, compression artifacts, and lighting flicker without excessive configuration. No universal "correct" threshold exists—0.20 is a statistically reasonable default that works across diverse content types.

Does the threshold affect video quality or just frame selection?

The threshold only affects which timestamps are selected, never the encoding quality of extracted frames. The resolution parameter (default 512px) controls output quality independently. A stricter threshold reduces output frame count; it doesn't degrade individual frame fidelity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}