# Near-Duplicate Frame Detection in Claude-Video: A Lightweight Perceptual Algorithm

> Discover Claude-Video's lightweight perceptual algorithm for near-duplicate frame detection. Learn how it efficiently identifies and removes identical frames to optimize video analysis.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-07-27

---

**Claude-Video detects near-duplicate frames by comparing 16×16 grayscale thumbnails using a mean absolute difference threshold of 2.0, greedily removing consecutive visually identical frames while preserving scene changes.**

The `bradautomates/claude-video` repository implements a fast, deterministic **near-duplicate frame detection** system that runs entirely within the Python standard library. This perceptual deduplication step activates after frame extraction to collapse static shots, duplicate slides, or terminal screenshots into a single representative frame, significantly reducing storage and processing overhead without requiring heavy computer vision dependencies.

## The Three-Stage Detection Pipeline

The algorithm operates in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) and consists of three distinct phases: thumbnail generation, pixel-level comparison, and greedy sequential filtering.

### Stage 1: Thumbnail Generation

First, the system downscales each extracted JPEG into a tiny grayscale thumbnail. The constants `DEDUP_THUMB = 16` and `DEDUP_THRESHOLD = 2.0` are defined at the module level in [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py) to control this process.

The helper function `_thumb_frames` invokes **ffmpeg** once to read all JPEG files, scales them to **16×16 pixels**, forces a gray pixel format, and returns the raw pixel bytes for every frame. This batch operation minimizes subprocess overhead while normalizing color information to luminance only.

```python

# From skills/watch/scripts/frames.py

DEDUP_THUMB = 16        # Thumbnail dimension (16x16)

DEDUP_THRESHOLD = 2.0   # Mean absolute difference threshold

```

### Stage 2: Pixel-Level Comparison

The function `_frame_delta` receives two thumbnail byte buffers and computes the **mean absolute difference** per pixel on a scale of 0–255. If the buffers differ in length—indicating mismatched extraction metadata—the function returns `inf` to prevent accidental deduplication of incompatible frames.

This metric avoids expensive perceptual hashing algorithms while reliably detecting visually identical content. The calculation is straightforward arithmetic on raw bytes, ensuring predictable performance regardless of input complexity.

### Stage 3: Greedy Sequential Filtering

The `dedupe_perceptual` function orchestrates the deduplication by calling `_dedupe_by_deltas`. The algorithm walks the chronological frame list and maintains a reference to the **last kept** thumbnail:

1. Keep the first frame as the initial reference.
2. For each subsequent frame, compare its thumbnail to the last kept thumbnail.
3. If the mean absolute difference is ≤ `DEDUP_THRESHOLD` (2.0), delete the JPEG file and omit the frame from the results.
4. If the difference exceeds the threshold, keep the frame and update the reference.

This greedy approach guarantees that only **consecutive** near-identical frames are merged, preserving rapid cuts and motion while eliminating static redundancies. After the pass completes, survivors are re-indexed and the function returns the count of dropped frames.

## Configuration and CLI Integration

The deduplication logic is tightly integrated with the extraction engines. The `extract`, `extract_scene_or_uniform`, and `extract_keyframes` functions all invoke `dedupe_perceptual` automatically unless the user supplies the `--no-dedup` flag.

In [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py), the CLI parser exposes this toggle:

```python

# From skills/watch/scripts/watch.py

parser.add_argument('--no-dedup', action='store_true',
                    help='Disable perceptual deduplication')

```

When enabled (the default), the pipeline ensures that the final frame set remains compact yet visually representative.

## Practical Usage Examples

### Programmatic Deduplication

You can invoke the deduplication logic directly in custom scripts after extracting frames:

```python
from pathlib import Path
from skills.watch.scripts.frames import dedupe_perceptual

# candidates is a list of frame dicts from any extraction function

candidates = [{'path': 'frame_0001.jpg', 'timestamp': 0.0}, 
              {'path': 'frame_0002.jpg', 'timestamp': 0.1}, ...]

unique_frames, dropped = dedupe_perceptual(candidates)

print(f"Kept {len(unique_frames)} frames, removed {dropped} near-duplicates")

```

### Command-Line Usage

Run the default extraction with automatic deduplication:

```bash
python3 -m skills.watch.scripts.watch video.mp4 out-dir

```

To preserve every raw frame without deduplication:

```bash
python3 -m skills.watch.scripts.watch video.mp4 out-dir --no-dedup

```

## Summary

- **Thumbnail-based comparison**: The algorithm reduces frames to 16×16 grayscale thumbnails using ffmpeg, minimizing memory and computational costs.
- **Mean absolute difference**: A threshold of 2.0 (on a 0–255 scale) determines visual similarity without requiring complex perceptual hashing libraries.
- **Greedy sequential filtering**: The algorithm compares each frame only to the last kept frame, ensuring that rapid scene changes are preserved while consecutive duplicates are removed.
- **Integrated pipeline**: Deduplication runs automatically after extraction in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) unless disabled via the `--no-dedup` flag in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py).

## Frequently Asked Questions

### What threshold does Claude-Video use for near-duplicate frame detection?

Claude-Video uses a **mean absolute difference threshold of 2.0** as defined by the `DEDUP_THRESHOLD` constant in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py). This value represents the average per-pixel difference on a 0–255 grayscale scale, effectively collapsing frames that differ only by compression artifacts or minor noise.

### How does the algorithm handle rapid scene changes?

The greedy sequential comparison in `_dedupe_by_deltas` compares each frame only to the **last kept** frame, not to all previous frames. This design ensures that rapid cuts—where consecutive frames differ significantly—are preserved in the output, while only sequences of visually identical frames are collapsed.

### Can I disable near-duplicate frame detection?

Yes. Pass the `--no-dedup` flag when running the CLI via [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py). This flag is exposed in the argument parser and respected by all extraction engines (`extract`, `extract_scene_or_uniform`, `extract_keyframes`), causing the pipeline to skip the `dedupe_perceptual` call entirely.

### Why use mean absolute difference instead of perceptual hashing?

The implementation prioritizes **speed and zero dependencies**. Mean absolute difference on 16×16 thumbnails requires only standard library arithmetic operations, whereas perceptual hashing libraries introduce additional dependencies and computational overhead. This approach reliably detects static shots and duplicate slides while maintaining deterministic, fast performance across platforms.