# How bradautomates/claude-video Handles Frame Deduplication Using Mean Absolute Difference

> Discover how bradautomates/claude-video deduplicates frames using Mean Absolute Difference. Learn about its efficient 16x16 thumbnail comparison technique.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-07-25

---

**bradautomates/claude-video removes visually duplicate frames by computing Mean Absolute Difference (MAD) between 16×16 grayscale thumbnails, keeping only frames that differ by more than 2.0 on a 0–255 scale.**

The `claude-video` repository implements a lightweight, pure-Python frame deduplication system that eliminates redundant video frames without external dependencies. By leveraging ffmpeg for thumbnail generation and Mean Absolute Difference for pixel-level comparison, the tool efficiently collapses static sequences while preserving distinct visual moments.

## The Three-Step MAD Deduplication Pipeline

The deduplication logic in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) operates through three tightly coupled stages that process candidate frames after initial extraction.

### Step 1: Generating 16×16 Grayscale Thumbnails

Before comparison, each candidate frame is downscaled to a tiny grayscale thumbnail to minimize computational overhead. In `frames._thumb_frames` (lines 424–460), the system invokes ffmpeg to convert extracted JPEGs into **16×16 pixel** grayscale arrays (`DEDUP_THUMB = 16`). This single-pass approach keeps the implementation within standard library constraints while producing 256-byte representations suitable for rapid comparison.

### Step 2: Computing Mean Absolute Difference

The core comparison occurs in `frames._frame_delta` (lines 315–322). For two thumbnail byte-arrays `a` and `b`, the function calculates the **Mean Absolute Difference** by summing absolute per-pixel differences and dividing by the total pixel count (256). This yields a normalized score in the range **0–255**, where identical frames produce 0.0 and maximally different frames approach 255.0.

### Step 3: Greedy Duplicate Removal

The deduplication algorithm in `frames._dedupe_by_deltas` (lines 380–400) implements a greedy selection strategy. Starting with the first candidate frame, each subsequent thumbnail is compared to the **last kept** frame using `_frame_delta`. If the MAD is **≤ `DEDUP_THRESHOLD` (2.0)**, the frame is considered a duplicate and deleted; otherwise, it becomes the new reference for subsequent comparisons. This approach ensures that only visually distinct moments survive while avoiding cascade effects that might unintentionally drop unique shots.

## Implementation Details in frames.py

The deduplication system exposes a clean public API through `frames.dedupe_perceptual` (lines 360–368). This function orchestrates the thumbnail generation and comparison pipeline, returning both the surviving frame list and the count of dropped duplicates.

Key implementation characteristics include:

- **Deterministic thresholding**: The hardcoded threshold of 2.0 specifically targets truly identical frames (static slides, frozen screens) while preserving genuine scene cuts
- **Memory efficiency**: Processing occurs on byte arrays rather than full-resolution images
- **Pure stdlib operation**: No computer vision libraries required; ffmpeg handles all image decoding

Higher-level extraction engines—including `extract_scene_or_uniform`, `extract_keyframes`, and `extract`—automatically invoke `dedupe_perceptual` when the `dedup=True` parameter is passed.

## Integration with Frame Extraction Engines

Frame deduplication operates as an optional post-processing step within the video analysis pipeline. After any extraction engine writes JPEG candidates to disk, the system checks the `dedup` parameter:

1. **Extraction**: Engines like `extract_scene_candidates` generate initial frame sets based on scene detection or uniform sampling
2. **Deduplication**: If enabled, `dedupe_perceptual` processes the candidate list through the MAD pipeline
3. **Capping**: The surviving frames may then be evenly sampled via `_even_sample` to respect `max_frames` limits

This workflow ensures that deduplication occurs before frame count constraints are applied, maximizing the diversity of the final selected frames.

## Code Examples

### Manual Deduplication of Frame Lists

```python
from pathlib import Path
import frames

# Assume candidates is a list of frame dicts from an extractor

candidates = [
    {"index": 0, "timestamp_seconds": 0.0, "path": "frame_0000.jpg", "reason": "scene-change"},
    {"index": 1, "timestamp_seconds": 0.1, "path": "frame_0001.jpg", "reason": "scene-change"},
]

# Run MAD-based deduplication with default threshold (2.0)

survivors, dropped = frames.dedupe_perceptual(candidates)

print(f"Kept {len(survivors)} frames, dropped {dropped} duplicates.")

```

### Enabling Deduplication in High-Level Extraction

```python
from pathlib import Path
import frames

video = "example.mp4"
out_dir = Path("frames_out")

# Extract using scene detection with automatic deduplication

frames_out, meta = frames.extract_scene_or_uniform(
    video_path=video,
    out_dir=out_dir,
    fps=2.0,
    target_frames=50,
    max_frames=100,
    dedup=True,  # Triggers MAD comparison

)

print("Engine:", meta["engine"])
print("Deduped frames:", meta["deduped_count"])

```

### Disabling Deduplication

```python
frames_out, meta = frames.extract_scene_or_uniform(
    video_path=video,
    out_dir=out_dir,
    fps=2.0,
    target_frames=50,
    max_frames=100,
    dedup=False,  # All frames retained regardless of similarity

)

```

## Testing and Validation

The repository includes comprehensive test coverage in [`tests/test_dedup.py`](https://github.com/bradautomates/claude-video/blob/main/tests/test_dedup.py) that verifies the MAD logic across edge cases:

- **`test_frame_delta_identical_is_zero`**: Confirms that identical thumbnails produce a delta of 0.0
- **`test_dedupe_collapses_identical_run`**: Validates that sequences of duplicate frames collapse to a single survivor
- **`test_dedupe_keeps_all_distinct`**: Ensures frames with large MAD values are fully retained
- **`test_dedupe_threshold_is_inclusive`**: Demonstrates that deltas exactly equal to 2.0 count as duplicates

These tests prove that the Mean Absolute Difference implementation correctly identifies visual redundancy without false positives.

## Summary

- **bradautomates/claude-video** implements frame deduplication using **Mean Absolute Difference** computed on 16×16 grayscale thumbnails
- The **greedy algorithm** compares each frame against the last kept frame, removing duplicates with MAD **≤ 2.0**
- Core functions reside in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py): `_thumb_frames` for generation, `_frame_delta` for comparison, and `_dedupe_by_deltas` for selection
- The system operates without external dependencies beyond ffmpeg, using pure-Python byte array operations
- Deduplication integrates seamlessly with all extraction engines via the `dedupe_perceptual` public API

## Frequently Asked Questions

### What threshold does claude-video use for frame deduplication?

The system uses a **MAD threshold of 2.0** on a 0–255 scale, defined as `DEDUP_THRESHOLD` in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py). This conservative value ensures only visually identical frames (such as static presentation slides or frozen video segments) are removed, while preserving subtle motion and scene transitions that exceed the threshold.

### Why does claude-video use 16×16 thumbnails for MAD calculation?

The **16×16 pixel resolution** (`DEDUP_THUMB = 16`) provides sufficient granularity to detect meaningful visual changes while minimizing computational overhead. At 256 bytes per thumbnail, the Mean Absolute Difference calculation requires only 256 subtraction and absolute value operations per comparison, enabling rapid processing of long video sequences without GPU acceleration or heavy dependencies.

### How does the greedy deduplication algorithm work?

The greedy approach implemented in `_dedupe_by_deltas` maintains a reference to the **last kept frame** and compares each subsequent candidate against this reference. If the MAD exceeds the threshold, the candidate becomes the new reference; otherwise, it is discarded. This ensures temporal coherence—if frame A differs from frame B, and B differs from C, all three are retained even if A and C might appear similar, preventing accidental loss of transitional content.

### Can I adjust the deduplication sensitivity in claude-video?

While the `DEDUP_THRESHOLD` constant is hardcoded to 2.0 in the source, advanced users can modify this value directly in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) (line 315) to adjust sensitivity. Lower values (such as 1.0) retain more frames by requiring stricter similarity, while higher values (such as 5.0) aggressively remove frames with minor variations. The repository test suite in [`tests/test_dedup.py`](https://github.com/bradautomates/claude-video/blob/main/tests/test_dedup.py) provides validation patterns for custom threshold calibration.