# How Near-Duplicate Frames Are Detected in Claude Video Using Thumbnails

> Discover how Claude Video efficiently detects near duplicate frames using 16x16 grayscale thumbnails. Learn the algorithm for frame removal based on pixel differences.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-08-02

---

**Claude Video detects near-duplicate frames by generating 16×16 grayscale thumbnails and greedily removing any frame whose mean absolute pixel difference to the previous kept frame is ≤ 2.0.**

The deduplication system in [bradautomates/claude-video](https://github.com/bradautomates/claude-video) provides a lightweight, perceptual approach to collapsing visually identical frames without relying on heavy computer vision models. This article breaks down the thumbnail-based detection algorithm implemented in the frame processing pipeline.

## Thumbnail Generation with FFmpeg

The detection pipeline begins in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) with the `_thumb_frames` function. This utility converts extracted JPEG frames into compact grayscale thumbnails through a single optimized FFmpeg command.

The command scales each frame to `DEDUP_THUMB × DEDUP_THUMB` (16 × 16 pixels) in grayscale format, outputting raw video bytes that are then split into individual thumbnail arrays.

```python

# From skills/watch/scripts/frames.py

DEDUP_THUMB = 16  # pixels

DEDUP_THRESHOLD = 2.0  # mean pixel difference threshold

```

Raw bytes from FFmpeg are parsed into one thumbnail per source frame, creating the compact representations used for all subsequent comparisons.

## Mean Pixel Difference Calculation

The `_frame_delta` function computes similarity between two thumbnails. It takes two thumbnail byte arrays `a` and `b` and returns the **mean absolute per-pixel difference** on a 0-255 scale.

If thumbnails differ in length—indicating malformed or mismatched data—the function returns infinity to ensure maximally different treatment.

```python

# Conceptual usage of _frame_delta

delta = _frame_delta(thumb_a, thumb_b)  # float in range [0, 255]

is_duplicate = delta <= DEDUP_THRESHOLD  # default: ≤ 2.0

```

This simple metric avoids cryptographic hashing pitfalls where visually similar frames produce wildly different hashes.

## The Greedy Deduplication Algorithm

The `dedupe_perceptual` function orchestrates the removal process. It operates as a **greedy chronological filter**: keep the first frame, then discard any subsequent frame whose thumbnail delta to the most recently kept frame falls at or below the threshold.

```python
from pathlib import Path
from skills.watch.scripts.frames import dedupe_perceptual

# Candidate frames from extraction pipeline

candidates = [
    {"index": 0, "timestamp_seconds": 0.0, "path": "out/frame_0000.jpg", "reason": "scene-change"},
    {"index": 1, "timestamp_seconds": 0.5, "path": "out/frame_0001.jpg", "reason": "scene-change"},
    # ...

]

# Run thumbnail-based deduplication

survivors, dropped = dedupe_perceptual(candidates)
print(f"Kept {len(survivors)} frames, dropped {dropped} near-duplicates.")

```

The helper `_dedupe_by_deltas` implements the core iteration logic:

1. Initialize with first candidate as "last kept"
2. For each subsequent candidate, compute thumbnail delta to last kept
3. If delta ≤ `DEDUP_THRESHOLD`, delete the JPEG file and continue
4. Otherwise, keep the frame and update "last kept"

Dropped frames are permanently removed from disk, and surviving frames receive reindexed metadata.

## Configuration and Tuning

Two constants control the detection behavior near the top of [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py):

| Constant | Value | Purpose |
|----------|-------|---------|
| `DEDUP_THUMB` | 16 | Thumbnail size in pixels (square) |
| `DEDUP_THRESHOLD` | 2.0 | Maximum mean pixel difference for duplicate classification |

The 16×16 resolution strikes a balance: large enough to preserve structural information, small enough for fast comparison. The threshold of 2.0 (roughly 0.8% of the 0-255 range) captures near-identical frames while preserving subtle but meaningful visual changes.

## Integration in Extraction Pipelines

Deduplication is **optional** and controlled via the `--no-dedup` CLI flag or `dedup=False` parameter. When enabled, it runs in three extraction modes as implemented in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py):

- **Uniform extraction** (`extract`): Frames deduplicated before return
- **Scene-based extraction** (`extract_scene_or_uniform`): Applied after scene detection
- **Key-frame extraction** (`extract_keyframes`): Near-identical keyframes collapsed

```bash

# Default: deduplication enabled

python -m skills.watch.scripts.watch https://youtube.com/... output_dir

# Disable to keep all extracted frames

python -m skills.watch.scripts.watch https://youtube.com/... output_dir --no-dedup

```

## Performance Characteristics

The thumbnail approach offers significant advantages over alternatives:

- **Speed**: Single FFmpeg pass for thumbnail generation, O(n) comparison loop
- **Memory**: 16×16 grayscale = 256 bytes per thumbnail versus full frames
- **Determinism**: Fixed thresholds produce reproducible results
- **Simplicity**: No neural network dependencies or model weights

The mean pixel difference metric intentionally trades sophistication for predictability—unlike learned perceptual hashes, behavior is fully inspectable and tunable via the threshold constant.

## Summary

- Claude Video detects near-duplicate frames using **16×16 grayscale thumbnails** generated via FFmpeg in `_thumb_frames`
- Similarity is measured by **mean absolute pixel difference** in `_frame_delta`, bounded by `DEDUP_THRESHOLD` (2.0)
- The **greedy chronological algorithm** in `dedupe_perceptual` keeps first frames and drops subsequent frames below threshold
- Deduplication is **configurable** via `--no-dedup` flag and applies across uniform, scene-based, and key-frame extraction modes
- All implementation lives in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) with integration points in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py)

## Frequently Asked Questions

### What resolution are the thumbnails used for duplicate detection?

Thumbnails are **16×16 pixels grayscale**, defined by the `DEDUP_THUMB = 16` constant in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py). This 256-byte representation is small enough for fast comparison while retaining enough structural information for perceptual similarity detection.

### How does the threshold value of 2.0 translate to visual similarity?

The threshold represents the **maximum mean absolute pixel difference** on a 0-255 scale. A value of 2.0 means average pixel values can differ by at most roughly 0.8% of full range—essentially capturing frames that are visually indistinguishable to human perception while preserving frames with any meaningful change.

### Can I disable near-duplicate detection when processing videos?

Yes. Pass the `--no-dedup` flag to the watch command, or set `dedup=False` when calling extraction functions programmatically. This preserves every extracted frame regardless of visual similarity to preceding frames.

### Why use mean pixel difference instead of perceptual hashing algorithms like pHash?

Mean pixel difference on tiny thumbnails provides **deterministic, inspectable behavior** without external dependencies. The Claude Video implementation prioritizes simplicity and speed over handling adversarial cases (resized, rotated, or heavily compressed duplicates) that production perceptual hashes address.