# How Scene-Change Detection Works in claude-video's Balanced vs. Token-Burner Modes

> Discover how claude-video's scene-change detection works in balanced vs. token-burner modes. Understand the differences and choose the right mode for your needs.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-08-04

---

**Scene-change detection in claude-video uses ffmpeg's scene filter to identify visual transitions, with balanced mode capping results at 100 frames and token-burner mode keeping every detected scene change. Both modes share identical detection logic but differ only in their `max_frames` parameter passed to `extract_scene_or_uniform`.**

claude-video, an open-source video analysis tool, offers two scene-aware processing modes: **balanced** and **token-burner**. This article explains how both leverage ffmpeg scene detection and where they diverge in frame sampling strategy, based on the source code in the `bradautomates/claude-video` repository.

## Scene-Change Detection Pipeline

Both modes execute the same four-stage pipeline defined in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py). The entry point `extract_scene_or_uniform` orchestrates detection, fallback, deduplication, and final sampling.

### FFmpeg Scene Filter Execution

The core detection happens in `extract_scene_candidates`, which constructs an ffmpeg video filter:

```python
vf = f"select='eq(n\\,0)+gt(scene\\,{threshold})',{_scale_filter(resolution)},showinfo"

```

This filter string performs three critical functions:

- `eq(n\,0)` — always selects the first frame (timestamp zero)
- `gt(scene, threshold)` — selects frames where ffmpeg's internal `scene` metric exceeds `SCENE_THRESHOLD = 0.20`
- `showinfo` — emits presentation timestamps for parsing candidate frames

The threshold of **0.20** (20% difference) filters out minor lighting changes while capturing genuine shot transitions.

### Uniform Sampling Fallback

If a video contains fewer than `SCENE_MIN_FRAMES = 8` scene cuts, the engine assumes static content (like screen recordings). It automatically falls back to simple uniform sampling via the `extract` function. This guarantees every video produces usable frames regardless of visual variety.

### Perceptual Deduplication

After raw candidates are collected, `dedupe_perceptual` processes them unless `--no-dedup` is specified. The function:

1. Generates **16×16 grayscale thumbnails** (`DEDUP_THUMB = 16`)
2. Computes mean-pixel difference between consecutive frames
3. Drops frames with difference ≤ **2.0** (`DEDUP_THRESHOLD = 2.0`)

This removes near-identical frames from the same shot while preserving distinct visual content.

## Balanced Mode: Capped Scene-Aware Sampling

Balanced mode targets typical use cases with a **100-frame ceiling**.

The configuration in [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py) defines:

```python
DEFAULT_DETAIL = "balanced"

def frame_cap(detail: str) -> Optional[int]:
    caps = {
        "low": 20,
        "medium": 50,
        "balanced": 100,
        "high": 200,
        "token-burner": None,  # uncapped

    }
    return caps.get(detail)

```

After deduplication, `_even_sample` distributes frames evenly: it always keeps the first and last frames, then spaces intermediates uniformly until reaching the cap. This ensures temporal coverage without clustering around high-activity regions.

**Invoke balanced mode:**

```bash
watch https://example.com/video.mp4 --detail balanced

```

Or programmatically:

```python
from pathlib import Path
from skills.watch.scripts.frames import extract_scene_or_uniform

frames, meta = extract_scene_or_uniform(
    video_path="myvideo.mp4",
    out_dir=Path("./out"),
    fps=1.0,                # ignored in scene mode

    target_frames=100,
    resolution=512,
    max_frames=100,         # cap enforces balanced behavior

    start_seconds=None,
    end_seconds=None,
    dedup=True,
)

```

## Token-Burner Mode: Uncapped Scene Extraction

Token-burner mode removes the ceiling entirely, returning **every deduplicated scene change** with no thinning.

The same `frame_cap` function returns `None` for this mode, which propagates to `_even_sample` as `max_frames=None`. The sampling function detects this and returns all frames without reduction.

This mode suits forensic analysis, animation review, or any scenario where missing a single shot transition carries high cost.

**Invoke token-burner mode:**

```bash
watch https://example.com/video.mp4 --detail token-burner

```

Or programmatically:

```python
from pathlib import Path
from skills.watch.scripts.frames import extract_scene_or_uniform

frames, meta = extract_scene_or_uniform(
    video_path="myvideo.mp4",
    out_dir=Path("./out"),
    fps=1.0,
    target_frames=100,      # ignored when max_frames=None

    resolution=512,
    max_frames=None,        # uncapped: all scene changes kept

    start_seconds=None,
    end_seconds=None,
    dedup=True,
)

```

## Mode Selection Logic in watch.py

The dispatch around line 214 in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) determines which mode activates:

```python
detail_budget = frame_cap(detail)  # 100 for balanced, None for token-burner

frames, frame_meta = extract_scene_or_uniform(
    video_path=video_path,
    out_dir=out_dir,
    fps=fps,
    target_frames=target_frames,
    resolution=resolution,
    max_frames=detail_budget,  # the only differing parameter

    start_seconds=start_seconds,
    end_seconds=end_seconds,
    dedup=not no_dedup,
)

```

Both branches call identical functions; only the `max_frames` argument changes. This architectural choice eliminates duplication while providing clear behavioral separation.

## Summary

- **Scene-change detection** uses ffmpeg's `scene` filter with a **0.20 threshold** in `extract_scene_candidates` ([`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py))
- **Balanced mode** caps results at **100 frames** via `_even_sample`, ensuring consistent token usage
- **Token-burner mode** sets `max_frames=None`, preserving **all detected scene changes** after deduplication
- Both modes rely on **perceptual deduplication** (16×16 thumbnails, difference threshold 2.0) to remove redundant frames
- The **uniform fallback** triggers for videos with fewer than 8 scene cuts, preventing empty results on static content

## Frequently Asked Questions

### How does claude-video decide between scene detection and uniform sampling?

The `extract_scene_or_uniform` function attempts scene detection first. If fewer than `SCENE_MIN_FRAMES = 8` candidates emerge, it automatically falls back to uniform sampling. This ensures screen recordings and static videos still yield representative frames.

### Can I adjust the scene detection sensitivity?

The threshold is hardcoded at `SCENE_THRESHOLD = 0.20` in [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py). Modifying this value requires editing the source; there is no command-line flag. Lower values increase sensitivity (more frames), while higher values reduce it.

### Why does token-burner mode still perform deduplication?

Deduplication runs unless `--no-dedup` is passed. Token-burner removes the **frame cap**, not redundancy filtering. Near-identical frames (mean difference ≤ 2.0) still collapse, preventing wasted tokens on visually duplicate content.

### What ffmpeg version is required for scene detection?

Any ffmpeg build with the `select` filter supports scene detection. The `scene` metric has been stable since ffmpeg 3.0 (2016). No external dependencies beyond standard ffmpeg are needed.