# How claude-video Implements Frame Deduplication Using Mean Absolute Difference

> Discover how claude-video eliminates duplicate frames using mean absolute difference. Learn how it compares thumbnails to keep only unique frames for efficient LLM processing.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-07-07

---

**claude-video removes visually duplicate frames before sending video to the LLM by comparing 16×16 grayscale thumbnails using mean absolute pixel difference, keeping only frames that differ by more than 2.0 average intensity units.**

The `claude-video` repository optimizes video analysis workflows by eliminating redundant visual information before it reaches the language model. This **frame deduplication mechanism using mean absolute difference** ensures that static scenes, fade transitions, and duplicate keyframes do not waste tokens or context window space. The implementation relies on lightweight image processing operations that remain fast and dependency-free by leveraging only Python's standard library alongside ffmpeg.

## The Three-Stage Deduplication Pipeline

The deduplication logic resides in **[`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py)** and executes in three distinct phases to minimize computational overhead while preserving visual distinctiveness.

### Stage 1: Generating 16×16 Grayscale Thumbnails

Each extracted JPEG is down-scaled to a **`16 × 16` pixel grayscale image** defined by the constant `DEDUP_THUMB = 16` (declared at lines 31-38). This aggressive reduction keeps the comparison operation fast and memory-efficient while retaining enough structural detail to distinguish between flat slides, fade transitions, and static screens.

The thumbnail generation occurs in **`_thumb_frames`** (lines 24-33), which invokes a single ffmpeg pass to read raw grayscale data directly into memory without creating intermediate files.

### Stage 2: Calculating Mean Absolute Pixel Difference

Similarity measurement happens in **`_frame_delta`** (lines 15-22). This function computes the **mean absolute difference** between corresponding pixels of two thumbnail arrays:

```python

# Conceptual implementation from frames.py lines 15-22

def _frame_delta(thumb_a, thumb_b):
    # Both are 16x16 uint8 arrays

    diff = np.abs(thumb_a.astype(float) - thumb_b.astype(float))
    return diff.mean()

```

If the returned delta is **≤ `DEDUP_THRESHOLD`** (default `2.0`), the frames are classified as "near-identical" and marked for removal. This threshold is defined alongside `DEDUP_THUMB` at the top of the file (lines 31-38).

### Stage 3: Greedy Chronological Filtering

The **`_dedupe_by_deltas`** function (lines 78-100) implements a greedy drop-while-keeping-last algorithm:

1. Keep the first frame as the initial reference.
2. Compare each subsequent frame's thumbnail to the **last kept** thumbnail.
3. If the delta exceeds the threshold, keep the frame and update the reference.
4. If the delta is within threshold, delete the JPEG file and skip the frame.

This approach ensures that the final sequence preserves the temporal structure while removing consecutive duplicates or extended static segments. Surviving frames are re-indexed to maintain sequential naming.

## Implementation Details in skills/watch/scripts/frames.py

The public entry point **`dedupe_perceptual`** orchestrates the workflow by first calling `_thumb_frames` to generate thumbnails for all candidates, then forwarding them to `_dedupe_by_deltas`. If fewer than two candidates exist or thumbnail generation fails, the function returns the original set unchanged.

The constants controlling sensitivity are co-located at the module level:

```python

# From skills/watch/scripts/frames.py lines 31-38

DEDUP_THUMB = 16        # Thumbnail dimensions (16x16)

DEDUP_THRESHOLD = 2.0   # Mean absolute difference cutoff

```

## Using the dedupe_perceptual API

You can invoke deduplication programmatically after extracting frames:

```python
from pathlib import Path
from skills.watch.scripts import frames

# Extract candidates using any engine (uniform, scene, keyframe)

candidates = frames.extract(
    video_path="example.mp4",
    out_dir=Path("out"),
    fps=1.0,
    max_frames=200,
)

# Run perceptual deduplication (mean absolute difference, threshold=2.0)

unique_frames, dropped = frames.dedupe_perceptual(candidates)

print(f"Kept {len(unique_frames)} frames, dropped {dropped} near-duplicates")

```

For command-line usage, the deduplication runs automatically unless disabled:

```bash

# CLI invocation (deduplication enabled by default)

python -m skills.watch.scripts.frames example.mp4 out --max-frames 200

# Disable deduplication explicitly

python -m skills.watch.scripts.frames example.mp4 out --max-frames 200 --no-dedup

```

Adjust the sensitivity by passing a custom threshold:

```python

# Stricter deduplication (keeps fewer frames)

unique_frames, dropped = frames.dedupe_perceptual(candidates, threshold=1.5)

# Looser deduplication (tolerates more noise)

unique_frames, dropped = frames.dedupe_perceptual(candidates, threshold=5.0)

```

## Integration with Video Processing Pipelines

The deduplication step is triggered automatically within every extraction engine—**uniform**, **scene**, and **keyframe**—unless the user passes the `--no-dedup` flag. After processing, [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) exposes the number of removed frames as **`deduped_count`** in the metadata output (lines 293-298), allowing you to verify how aggressively the pipeline compressed the video.

Post-deduplication, the remaining frames may undergo further sampling (such as `_even_sample`) before being encoded for the LLM.

## Summary

- **claude-video** collapses redundant frames using a **mean absolute difference** metric computed on `16×16` grayscale thumbnails.
- The **`_frame_delta`** function (lines 15-22) calculates average pixel differences, while **`_dedupe_by_deltas`** (lines 78-100) performs greedy chronological filtering against the most recently kept frame.
- The default **`DEDUP_THRESHOLD = 2.0`** (defined at lines 31-38) balances sensitivity and compression.
- Access the functionality via **`dedupe_perceptual`** in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) or disable it with `--no-dedup` in the CLI.
- The pipeline reports the number of dropped frames as **`deduped_count`** in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) (lines 293-298).

## Frequently Asked Questions

### Why does claude-video use mean absolute difference instead of perceptual hashing?

Mean absolute difference provides deterministic, lightweight comparison that requires no additional dependencies beyond numpy and standard library operations. According to the source code in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), this approach avoids the complexity of cryptographic or perceptual hashing libraries while remaining sufficiently accurate to detect static screens and fade transitions on low-resolution thumbnails.

### What happens if thumbnail generation fails during deduplication?

If `_thumb_frames` (lines 24-33) fails to extract grayscale data from the input JPEGs, `dedupe_perceptual` returns the original candidate list unchanged and sets `dropped` to zero. This fail-safe ensures that extraction pipelines continue operating even when processing corrupted or unsupported image formats.

### How does the greedy algorithm handle sudden scene changes?

The **`_dedupe_by_deltas`** function (lines 78-100) compares each candidate only to the **last kept** frame rather than the immediate predecessor. This means that after a scene change, the new distinct frame becomes the reference, and subsequent similar frames are compared against this new reference. This prevents false positives when transitioning between two different but static scenes.

### Can I adjust the frame deduplication threshold for noisy video content?

Yes. Pass a custom `threshold` parameter to `dedupe_perceptual`. Values higher than the default `2.0` (such as `5.0`) tolerate more pixel variation, useful for noisy or low-light footage, while values lower than `2.0` enforce stricter deduplication for high-fidelity content. The threshold represents the mean absolute intensity difference across the `16×16` thumbnail grid.