# How Claude Video's Frame Deduplication Algorithm Works: Perceptual Hashing for Video Processing

> Discover how Claude Video's frame deduplication algorithm uses perceptual hashing to remove near-duplicate frames. Learn about its efficient downscaling and difference calculation for video processing.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: internals
- Published: 2026-07-28

---

**Claude Video removes near-duplicate frames by downscaling images to 16×16 grayscale thumbnails and calculating mean absolute differences against the last kept frame, preserving only frames that exceed a configurable similarity threshold.**

The bradautomates/claude-video repository implements an efficient perceptual deduplication system that preprocesses video content before the frame budget cap is applied. This **Claude Video frame deduplication algorithm** ensures that only visually distinct frames are sent to the Claude API, significantly reducing token costs for videos containing static segments, slide decks, or paused recordings.

## Core Algorithm Components

The deduplication pass operates as a lightweight preprocessing step that trades minimal CPU overhead for substantial reductions in redundant data.

### Thumbnail Generation and Normalization

Every extracted frame undergoes aggressive downscaling to enable rapid comparison. In [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), the `thumb_image` helper (around line 440) generates a square thumbnail of `DEDUP_THUMB` pixels—defaulting to **16×16**—and converts the image to 8-bit grayscale. This normalization drastically reduces the data footprint while preserving the essential visual "shape" required for similarity detection.

### Per-Pixel Distance Calculation

For each candidate frame, the algorithm computes the **mean absolute difference** between the current frame's thumbnail and the thumbnail of the last kept frame. This produces a simple scalar distance value that quantifies visual dissimilarity without requiring complex perceptual hashing algorithms.

### Greedy Last-Kept Comparison Logic

The core selection logic resides in the private function **`_dedupe_by_deltas`** (defined at line 479 of [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py)). This function implements a greedy filtering strategy:

- A frame is **kept only if** its distance exceeds the configurable `threshold` (default **2.0**)
- Comparison is always performed against the *last kept* frame rather than the immediate predecessor
- This design collapses entire runs of identical or near-identical frames into a single representative image

The function returns two values: a list of surviving frame timestamps and the total count of dropped frames.

## Pipeline Integration and CLI Control

The deduplication step integrates seamlessly with the main extraction workflow while remaining optional for specialized use cases.

### Invocation and Flags

The deduplication pass is invoked by the main extraction pipeline through the sequence `extract_frames` → `dedupe_perceptual`. Control flags are parsed in **[`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py)** (see lines 212 and 224), where the `--no-dedup` flag allows users to bypass deduplication entirely when every sampled frame is required for fine-grained motion analysis.

### Metadata Tracking

After processing, the pipeline records the effectiveness of the deduplication pass by adding **`deduped_count`** to the output metadata (around line 549 of [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py)). This metric allows users to verify how many redundant frames were eliminated before the frame budget was applied.

## Practical Usage Examples

Run the watch skill with default deduplication enabled (recommended for most content):

```bash
python -m skills.watch.scripts.watch https://www.youtube.com/watch?v=example

```

Disable deduplication when processing content requiring frame-perfect analysis:

```bash
python -m skills.watch.scripts.watch https://www.youtube.com/watch?v=example --no-dedup

```

Call the deduplication routine directly in custom pipelines:

```python
from skills.watch.scripts import frames

# candidates: list of (timestamp, path) tuples from the extractor

# thumbs: list of 16×16 greyscale thumbnails for those frames

survivors, dropped = frames._dedupe_by_deltas(
    candidates, 
    thumbs, 
    threshold=2.0
)

print(f"Kept {len(survivors)} frames, dropped {dropped} near-duplicates")

```

## Performance Impact and Budget Optimization

Because the **Claude Video frame deduplication algorithm** runs before the `--max-frames` cap is enforced, the remaining budget is spent exclusively on **distinct visual content**. This architecture delivers particular value for:

- **Screen recordings** with static UI elements
- **Slide deck presentations** with extended pauses on individual slides
- **Paused or buffering video segments** where content remains unchanged

The computational cost is negligible—a single downscale operation and mean-difference calculation per frame—while the savings in API tokens and processing time can be substantial for content with high temporal redundancy.

## Summary

- **Thumbnail preprocessing**: Frames are downscaled to 16×16 grayscale to enable rapid comparison while preserving visual structure.
- **Similarity metric**: Mean absolute difference between thumbnails determines frame uniqueness.
- **Greedy filtering**: The `_dedupe_by_deltas` function in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) keeps frames only when they differ significantly from the last kept frame (threshold default 2.0).
- **Optional execution**: The `--no-dedup` flag in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) allows bypassing the filter when complete frame sequences are required.
- **Budget efficiency**: Deduplication occurs before the `--max-frames` limit, ensuring the frame budget consumes only visually distinct content.

## Frequently Asked Questions

### What is the default similarity threshold for frame deduplication?

The default threshold is **2.0**, representing the mean absolute difference between 16×16 grayscale thumbnails. You can configure this value when calling `frames._dedupe_by_deltas()` directly, though the CLI uses this default consistently.

### How do I disable frame deduplication when processing a video?

Pass the `--no-dedup` flag when invoking the watch skill. This flag is parsed in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) (lines 212 and 224) and causes the pipeline to skip the `dedupe_perceptual` step entirely, preserving every sampled frame for analysis.

### Why does the algorithm compare against only the last kept frame instead of all previous frames?

Comparing against the **last kept frame** (rather than the immediate predecessor or all previous frames) creates a greedy collapsing behavior. When a sequence of near-identical frames appears, this approach selects the first distinct frame and drops all subsequent similar frames until a significant change occurs. This is more computationally efficient than pairwise comparison against all history while still effectively eliminating redundant static sequences.

### Does deduplication affect the maximum frame budget calculation?

Yes, deduplication runs **before** the `--max-frames` budget cap is applied. Because duplicate frames are removed prior to budget enforcement, the final frame count sent to Claude represents only visually distinct content. This means a video with 1000 extracted frames but only 50 distinct visual states will consume only 50 frames from your budget after deduplication.