# How bradautomates/claude-video Performs Frame Deduplication Using Grayscale Thumbnails and Mean Absolute Difference

> Discover how bradautomates/claude-video performs frame deduplication. It uses grayscale thumbnails and mean absolute difference to efficiently identify and remove redundant video frames.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-07-30

---

**bradautomates/claude-video detects near-duplicate video frames by converting each extracted JPEG into a 16×16 grayscale thumbnail and computing the mean absolute per-pixel difference between successive thumbnails, removing frames where the difference is less than or equal to 2.0.**

The `claude-video` repository implements a fast perceptual deduplication pipeline that processes video frames through ffmpeg-generated thumbnails and deterministic comparison algorithms. This system reduces redundant visual data before AI analysis by scaling high-resolution frames into compact grayscale representations and applying a greedy mean-absolute-difference strategy to identify true visual duplicates.

## Grayscale Thumbnail Generation with ffmpeg

The deduplication process begins in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) with the `_thumb_frames()` function (lines 24-31), which generates compact representations of each extracted frame. The constant `DEDUP_THUMB = 16` defines a fixed 16×16 pixel thumbnail size that balances computational efficiency with structural detail.

The function constructs a single ffmpeg command (lines 48-50) that reads the JPEG sequence, applies the `scale=16:16` filter, forces the `gray` pixel format, and streams raw video bytes back to Python. The raw byte stream is split into a list of `bytes` objects—one per frame—each containing exactly `DEDUP_THUMB × DEDUP_THUMB` grayscale values representing the luminance structure of the original image.

## Mean Absolute Difference Calculation

Once thumbnails are generated, the `_frame_delta()` function (lines 15-22 in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py)) calculates the similarity between two frame buffers using mean absolute difference (MAD). This metric computes the average per-pixel absolute distance between two thumbnail byte buffers on a scale of 0-255.

```python
def _frame_delta(a: bytes, b: bytes) -> float:
    # mean absolute per‑pixel difference (0‑255)

    if not a or len(a) != len(b):
        return float("inf")
    return sum(abs(x - y) for x, y in zip(a, b)) / len(a)

```

The function includes a critical safety guard: it returns `float("inf")` if the input buffers differ in length or are empty. This ensures that corrupted or mismatched frames are never incorrectly collapsed as duplicates, preventing data corruption in the deduplication pipeline.

## Greedy Deduplication Algorithm

The `dedupe_perceptual()` function (lines 63-70) orchestrates the deduplication workflow by first generating thumbnails for all candidate frames and then invoking `_dedupe_by_deltas()` (lines 78-86) to perform the actual duplicate detection.

The algorithm uses a greedy chronological approach based on the constant `DEDUP_THRESHOLD = 2.0`. It keeps the first frame as a reference, then compares each subsequent thumbnail against the **last kept frame** using `_frame_delta()`. If the mean absolute difference is less than or equal to the threshold, the candidate frame is marked as a duplicate and scheduled for deletion; otherwise, it becomes the new reference frame.

```python
def _dedupe_by_deltas(candidates, thumbs, threshold=DEDUP_THRESHOLD):
    kept = [candidates[0]]
    last = thumbs[0]
    dropped = []
    for cand, thumb in zip(candidates[1:], thumbs[1:]):
        if _frame_delta(thumb, last) <= threshold:
            dropped.append(cand)
        else:
            kept.append(cand)
            last = thumb
    # cleanup: delete dropped JPEGs & re‑index survivors

```

After processing, the function deletes the dropped JPEGs from disk and re-indexes the survivors to maintain sequential naming.

## Integration Across Extraction Engines

The deduplication system integrates with every frame extraction engine in the repository, including `extract()`, `extract_scene_candidates()`, and `extract_keyframes()`. Unless the user passes the `--no-dedup` CLI flag via [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py), the system automatically invokes `dedupe_perceptual()` on the extracted frame candidates.

The deduplication results are reported through the `deduped_count` field in the JSON metadata returned by the CLI, providing transparency about how many redundant frames were removed from the analysis set.

## Code Examples

To deduplicate frames programmatically after extraction:

```python
from skills.watch.scripts.frames import dedupe_perceptual

candidates = [
    {"path": "frame_0001.jpg", "timestamp_seconds": 0.0, "reason": "uniform"},
    {"path": "frame_0002.jpg", "timestamp_seconds": 0.5, "reason": "uniform"},
    # … additional frames …

]

# Uses default threshold of 2.0

unique_frames, n_dropped = dedupe_perceptual(candidates)

print(f"Kept {len(unique_frames)} frames, removed {n_dropped} duplicates")

```

From the command line, deduplication runs automatically:

```bash
python -m skills.watch.scripts.frames video.mp4 output_dir --max-frames 200

```

The output metadata includes `"deduped_count": <n>` indicating the number of duplicates removed.

## Summary

- **Grayscale thumbnails**: The `_thumb_frames()` function in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) generates 16×16 pixel thumbnails using ffmpeg's `scale` and `gray` format filters.
- **MAD calculation**: `_frame_delta()` computes mean absolute per-pixel difference between thumbnail buffers, returning infinity for size mismatches to prevent data corruption.
- **Greedy deduplication**: `_dedupe_by_deltas()` compares each frame against the last kept frame using a default threshold of 2.0, removing duplicates when the difference falls below this value.
- **Transparent reporting**: The system reports removed frames via `deduped_count` in the CLI metadata and can be disabled with the `--no-dedup` flag.

## Frequently Asked Questions

### What is the default threshold for frame deduplication in claude-video?

The default threshold is **2.0**, defined by the `DEDUP_THRESHOLD` constant in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py). Frames with a mean absolute difference of 2.0 or less (on a 0-255 per-pixel scale) are considered visual duplicates and removed from the candidate set. This conservative threshold ensures that only nearly identical frames are eliminated while preserving meaningful visual changes.

### Why does claude-video use 16×16 grayscale thumbnails instead of full-resolution frames?

The `DEDUP_THUMB = 16` constant specifies a 16×16 pixel resolution because processing full-resolution frames would be computationally expensive without improving deduplication accuracy. The grayscale conversion eliminates color channel noise while preserving the structural luminance information necessary to detect true visual duplicates. This approach processes video data through compact byte buffers that require minimal memory compared to full JPEG decoding.

### How does the algorithm handle corrupted frames or size mismatches?

The `_frame_delta()` function includes explicit guards that return `float("inf")` when input buffers are empty or differ in length. This ensures that corrupted frames, partial reads, or resolution changes are never incorrectly collapsed as duplicates. The greedy deduplication algorithm treats these infinity values as non-matches, forcing the system to keep potentially corrupted frames rather than risk data loss through erroneous deletion.

### Can I disable frame deduplication or adjust the threshold?

You can disable deduplication entirely by passing the `--no-dedup` flag to the CLI in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py). However, the threshold is hardcoded to 2.0 in the current implementation via `DEDUP_THRESHOLD`. To use a different sensitivity, you would need to modify the constant in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) or call `dedupe_perceptual(candidates, threshold=your_value)` directly from Python when using the programmatic API.