# How Perceptual Frame Deduplication Works in Claude-Video

> Learn how perceptual frame deduplication in Claude-Video eliminates redundant frames using thumbnail comparisons and a greedy algorithm. Discover the 2.0 pixel difference threshold.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-08-09

---

**Claude-Video removes near-duplicate frames by comparing 16×16 grayscale thumbnails using a greedy algorithm that drops frames when the mean absolute pixel difference falls below a threshold of 2.0.**

Perceptual frame deduplication in the `bradautomates/claude-video` repository eliminates redundant video frames before LLM processing, reducing token costs while preserving visual information. The system analyzes tiny **perceptual thumbnails** rather than full-resolution images to detect near-identical content efficiently. This implementation uses a single ffmpeg pass for thumbnail generation followed by a deterministic greedy filter.

## Thumbnail Generation for Perceptual Deduplication

The deduplication pipeline begins by creating compact perceptual representations of each extracted frame. In [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), the **`_thumb_frames`** function (lines 24-31) orchestrates a single ffmpeg pass that downscales every JPEG to a **16×16 grayscale image** (`DEDUP_THUMB = 16`).

This process returns a list of raw byte buffers, one per frame, minimizing memory overhead compared to loading full-resolution images. The thumbnail dimensions are deliberately small—just 256 total pixels—making subsequent comparisons computationally trivial while retaining enough structural information to detect meaningful visual changes.

## Measuring Frame Similarity with Mean Absolute Difference

Once thumbnails are generated, the system quantifies visual differences using the **`_frame_delta`** function (lines 15-22). This utility computes the **mean absolute per-pixel difference** between two thumbnail buffers.

If the two buffers differ in length—indicating corruption or mismatched extraction—the function returns **infinite distance**. This safety mechanism ensures that malformed frames are never collapsed into valid sequences, preserving data integrity during the deduplication process.

## The Greedy Perceptual Deduplication Algorithm

The core logic resides in **`dedupe_perceptual`** and its helper **`_dedupe_by_deltas`** (lines 63-71 and 78-89). The algorithm operates as follows:

1. Initialize with the first frame as the "last kept" reference.
2. Iterate through remaining frames, comparing each to the last kept thumbnail.
3. Drop the current frame if its delta is ≤ **`DEDUP_THRESHOLD`** (default **2.0**).
4. Keep the frame and update the reference if the delta exceeds the threshold.

This greedy approach efficiently collapses static or near-static periods while guaranteeing that the first occurrence of any visual state is preserved. When frames are dropped, their corresponding JPEG files are deleted from disk, and the surviving frames are re-indexed to maintain sequential order. The deduplication logic is validated by unit tests in [`tests/test_dedup.py`](https://github.com/bradautomates/claude-video/blob/main/tests/test_dedup.py), which verify that identical frames are collapsed while distinct frames are preserved.

## Integration with Frame Extraction Engines

Perceptual frame deduplication applies universally across all frame extraction strategies—including scene-change detection, uniform sampling, and keyframe extraction—unless explicitly disabled. The CLI exposes a **`--no-dedup`** flag (lines 165-170) to bypass this step for debugging or specific analysis requirements.

During processing, the `extract_scene_or_uniform` and `extract_keyframes` functions (lines 44-57 and 65-68) invoke the deduplication routine automatically. The system tracks efficiency by recording the number of removed frames in the metadata dictionary under **`deduped_count`**, allowing downstream consumers to audit compression ratios. End-to-end tests in [`tests/test_watch.py`](https://github.com/bradautomates/claude-video/blob/main/tests/test_watch.py) confirm that this metadata is reported correctly.

## Implementation Example

To deduplicate frames programmatically using the default threshold:

```python
from pathlib import Path
from skills.watch.scripts import frames

# Assume `candidates` is a list of dicts produced by any extraction engine:

#   [{"path": "frame_0001.jpg", "timestamp_seconds": 0.0, ...}, …]

# Perform perceptual deduplication (default threshold = 2.0)

deduped_frames, dropped = frames.dedupe_perceptual(candidates)

print(f"Removed {dropped} near‑duplicate frames")
for f in deduped_frames:
    print(f["timestamp_seconds"], f["path"])

```

To process a video without deduplication via the command line:

```bash
python -m skills.watch.scripts.frames video.mp4 out_dir --no-dedup

```

## Summary

- **Perceptual frame deduplication** in Claude-Video uses 16×16 grayscale thumbnails generated via ffmpeg to minimize computational overhead.
- The **mean absolute pixel difference** metric quantifies similarity, with infinite distance returned for buffer mismatches to prevent data corruption.
- A **greedy algorithm** retains the first frame of each visual sequence and drops subsequent frames with deltas ≤ 2.0.
- The system integrates with all extraction engines and tracks removed frames via the **`deduped_count`** metadata field.

## Frequently Asked Questions

### What threshold does Claude-Video use for perceptual deduplication?

The default **`DEDUP_THRESHOLD`** is **2.0**, representing the maximum mean absolute pixel difference between 16×16 thumbnails before frames are considered distinct. This tight threshold ensures only virtually identical frames are collapsed, preserving meaningful visual changes while eliminating static redundancy.

### How does the algorithm handle corrupted or mismatched frame buffers?

The **`_frame_delta`** function returns **infinite distance** when comparing buffers of differing lengths. According to the source code in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) (lines 15-22), this prevents mismatched or corrupted frames from being incorrectly merged with valid frames, ensuring data integrity throughout the pipeline.

### Can I disable perceptual frame deduplication when processing videos?

Yes. Pass the **`--no-dedup`** flag when invoking the CLI, as implemented in lines 165-170 of [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py). This bypasses the `dedupe_perceptual` call entirely, retaining all extracted frames regardless of visual similarity. Disabling deduplication is useful for debugging or when analyzing frame-level metadata without compression.

### What happens to frames that are removed during deduplication?

Dropped frames are permanently deleted from the output directory during the re-indexing phase. The **`_dedupe_by_deltas`** function (lines 78-89) removes the underlying JPEG files and compacts the remaining frames into a continuous sequence. The count of deleted frames is stored in the **`deduped_count`** metadata field for audit purposes.