# How Scene-Change Detection Works with SCENE_THRESHOLD and When Uniform Sampling Kicks In

> Discover how claude-video's scene-change detection uses SCENE_THRESHOLD and automatically switches to uniform sampling when fewer than 8 scene changes occur. Optimize your video analysis.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-08-10

---

**Scene-change detection in `claude-video` uses FFmpeg's scene filter with a `SCENE_THRESHOLD` of 0.20 to detect visual cuts, and automatically falls back to uniform sampling when fewer than 8 scene changes are detected.**

The `bradautomates/claude-video` repository implements an adaptive keyframe extraction system that prioritizes meaningful scene transitions over arbitrary timestamps. This article explains the mechanics of scene-change detection, the role of the `SCENE_THRESHOLD` constant, and the precise conditions that trigger uniform sampling.

## What is SCENE_THRESHOLD and How It Detects Scene Changes

The `SCENE_THRESHOLD` constant defines the sensitivity of FFmpeg's scene-change detection filter. As implemented in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) (line 20), the default value is **0.20**:

```python
SCENE_THRESHOLD: float = 0.20

```

This threshold is passed to the `extract_scene_candidates` function, which constructs an FFmpeg video filter expression (lines 224–226):

```python
vf = f"select='eq(n\\,0)+gt(scene\\,{threshold})',{_scale_filter(resolution)},showinfo"

```

The filter components break down as follows:

- `eq(n,0)` — always select the first frame as a baseline reference
- `gt(scene,{threshold})` — select frames where the scene-change score exceeds the threshold
- `_scale_filter(resolution)` — downscale frames for efficient processing
- `showinfo` — emit metadata about selected frames

FFmpeg computes the **scene metric** by comparing histogram differences between consecutive frames. When this difference exceeds `SCENE_THRESHOLD`, FFmpeg emits that frame as a candidate keyframe. Higher thresholds yield fewer, more dramatic scene changes; lower thresholds capture subtle transitions.

## When the System Falls Back to Uniform Sampling

Scene-change detection alone cannot guarantee sufficient coverage for all video types. The `extract_scene_or_uniform` function (lines 510–543) implements a reliability check:

```python
def extract_scene_or_uniform(
    video_path: Path,
    out_dir: Path,
    max_frames: Optional[int] = None,
    dedupe: bool = True,
) -> Tuple[List[Path], Dict]:
    # ... scene detection attempt ...

    scene_frames = extract_scene_candidates(...)
    scene_count = len(scene_frames)
    
    if scene_count >= SCENE_MIN_FRAMES:  # SCENE_MIN_FRAMES = 8

        return scene_frames, {"engine": "scene", "count": scene_count}
    
    # Fall back to uniform sampling

    timestamps = _generate_uniform_timestamps(video_path, target_count=8)
    uniform_frames = extract_at_timestamps(video_path, timestamps, out_dir)
    return uniform_frames, {"engine": "uniform", "count": len(uniform_frames)}

```

The fallback condition is straightforward: **if fewer than `SCENE_MIN_FRAMES` (8) scene changes are detected**, the system switches to uniform sampling. This handles:

- Static videos with minimal visual change
- Slow-motion or single-shot footage
- Screen recordings with gradual transitions

## The Uniform Sampling Implementation

Uniform sampling distributes frames evenly across the video duration. The `extract_at_timestamps` function receives precomputed timestamps and extracts frames at those specific points, ensuring consistent coverage regardless of visual content.

Key differences between the two engines:

| Engine | Selection Criteria | Use Case |
|--------|-------------------|----------|
| **Scene** | Visual difference > 0.20 | Videos with distinct shots, dialogue cuts, camera movements |
| **Uniform** | Evenly spaced timestamps | Static content, slideshows, single-take recordings |

## Practical Code Examples

### Inspecting the Threshold and Engine Selection

```python
from skills.watch.scripts.frames import (
    SCENE_THRESHOLD,
    SCENE_MIN_FRAMES,
    extract_scene_or_uniform,
)
from pathlib import Path

# Check default constants

print(f"Scene threshold: {SCENE_THRESHOLD}")      # 0.20

print(f"Minimum scene frames: {SCENE_MIN_FRAMES}")  # 8

# Extract frames with automatic engine selection

frames, metadata = extract_scene_or_uniform(
    video_path=Path("interview.mp4"),
    out_dir=Path("output/"),
    max_frames=16,
)

print(f"Engine used: {metadata['engine']}")
print(f"Frames extracted: {metadata['count']}")

```

### Handling a Static Video (Forces Uniform Sampling)

```python

# A screen recording with no cuts will trigger uniform sampling

frames, metadata = extract_scene_or_uniform(
    video_path=Path("static_recording.mp4"),
    out_dir=Path("uniform_output/"),
)

assert metadata["engine"] == "uniform"
assert metadata["count"] >= 8  # Guaranteed minimum coverage

```

## Key Files and Functions in the Repository

- [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) — Core implementation containing `SCENE_THRESHOLD`, `extract_scene_candidates()`, and `extract_scene_or_uniform()`
- [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) — CLI entry point that orchestrates frame extraction based on the `--detail` flag
- [`tests/test_frames.py`](https://github.com/bradautomates/claude-video/blob/main/tests/test_frames.py) — Test coverage validating scene engine selection and uniform fallback behavior

## Summary

- **SCENE_THRESHOLD (0.20)** controls FFmpeg's scene-change filter sensitivity—higher values detect only major cuts, lower values capture subtle transitions
- **Scene detection runs first** via `extract_scene_candidates()`, which builds an FFmpeg filter selecting frames where `scene > threshold`
- **Uniform sampling triggers automatically** when fewer than 8 scene changes are detected, ensuring minimum frame coverage for downstream processing
- The **engine metadata** field (`"scene"` or `"uniform"`) records which method was used for each extraction

## Frequently Asked Questions

### How do I make scene detection more sensitive?

Lower the threshold by modifying `SCENE_THRESHOLD` in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) or passing a custom value to `extract_scene_candidates()`. Values below 0.15 detect more subtle transitions but may include false positives from minor lighting changes.

### Why 8 frames as the minimum for scene detection?

The `SCENE_MIN_FRAMES` constant (line 19 in [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py)) ensures sufficient visual diversity for typical downstream tasks like video summarization or CLIP-based analysis. This value balances coverage against redundancy.

### Can I force uniform sampling even for videos with many scene changes?

Currently `extract_scene_or_uniform()` does not expose a force-uniform parameter. You can call `extract_at_timestamps()` directly with custom timestamps, or modify the threshold logic to always fail the scene count check.

### Does the threshold value affect processing speed?

No—FFmpeg evaluates the scene metric for every frame regardless of threshold. The threshold only filters which frames are emitted. Processing time depends primarily on video resolution and the `_scale_filter()` downscaling factor.