# How Frame Deduplication Works in Claude's Watch Skill

> Discover how Claude's watch skill achieves frame deduplication using thumbnails, pixel difference calculations, and an O(n) filter to optimize video compression and preserve key moments.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: internals
- Published: 2026-07-17

---

**The watch skill removes visually identical frames by generating 16×16 grayscale thumbnails, calculating mean pixel differences, and applying a greedy O(n) filter that drops frames below a threshold of 2.0 to compress static sequences while preserving distinct moments.**

The watch skill in the `bradautomates/claude-video` repository extracts representative frames from video content, but raw extraction often yields redundant images from static scenes or slow transitions. Frame deduplication solves this by collapsing near-duplicate frames into single representatives, keeping the output compact without sacrificing visual diversity. This article explains the perceptual deduplication pipeline implemented in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) and controlled via the `--no-dedup` CLI flag.

## The Three-Stage Deduplication Pipeline

After initial frame extraction, the deduplication process converts high-resolution JPEGs into comparable fingerprints and filters them chronologically using three coordinated functions.

### Thumbnail Generation with FFmpeg

The `_thumb_frames` function in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) (lines 24-41) downsizes each candidate frame to a tiny 16×16 grayscale thumbnail using a single `ffmpeg` pass. This resolution, defined by the constant `DEDUP_THUMB = 16`, reduces computational overhead while preserving enough structural information to detect true duplicates. The implementation remains pure-stdlib compatible, processing the byte output from `ffmpeg` without external image libraries.

### Mean-Pixel Difference Calculation

Once thumbnails exist, the `_frame_delta` function (lines 15-22) computes the average absolute per-pixel difference between two thumbnail byte arrays on a 0-255 scale. If the mean difference falls below `DEDUP_THRESHOLD = 2.0`—approximately 0.8% average change per pixel—the frames are considered near-identical and marked for removal. This conservative threshold ensures only virtually identical frames (such as duplicate frames in static screen recordings) are collapsed, while preserving fast-moving slides or scrolling terminals.

### Greedy Chronological Filtering

The `_dedupe_by_deltas` function (lines 79-87) implements a greedy algorithm that walks the chronological candidate list in O(n) time. It keeps the first frame and compares each subsequent thumbnail only to the **last kept** frame. When the delta exceeds the threshold, the frame survives and becomes the new comparison baseline; otherwise, the JPEG file is deleted immediately. This approach respects visual continuity—static segments collapse to single frames while genuine changes (even subtle slide transitions) break the chain because the delta exceeds the threshold.

## Public API and Integration

### The dedupe_perceptual Entry Point

The `dedupe_perceptual` function (lines 63-71) serves as the public interface used by extraction engines throughout the skill. It guards against trivial inputs (`len(candidates) <= 1`), orchestrates thumbnail generation via `_thumb_frames`, and delegates to `_dedupe_by_deltas`. The function returns a tuple of `(surviving_candidates, dropped_count)`, allowing callers to report compression statistics.

### CLI Integration and Metadata

In [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) (lines 212-224), the `--no-dedup` flag toggles the `dedup` boolean passed to `extract_keyframes` and `extract_scene_or_uniform`. When deduplication is enabled, it runs after frame extraction, and engines record `deduped_count` in result metadata. The final report displays the count of dropped frames, such as "2 near-duplicate(s) dropped."

## Configuration and Threshold Selection

The `DEDUP_THRESHOLD = 2.0` value is deliberately conservative. On the 0-255 pixel value scale, this threshold only collapses frames that are virtually identical, preventing the loss of meaningful content in fast-moving videos while effectively compressing redundant static sequences.

## Fail-Open Safety Mechanisms

If thumbnail creation fails due to `ffmpeg` errors or byte count mismatches, `_thumb_frames` returns an empty list, causing `dedupe_perceptual` to return the original candidates unchanged. This ensures the extraction pipeline never crashes because of deduplication failures, maintaining availability over strict deduplication.

## Code Examples

Run the watch skill with default deduplication enabled (removes near-duplicates):

```bash
watch https://example.com/video.mp4

```

Disable deduplication to keep every extracted frame for debugging:

```bash
watch https://example.com/video.mp4 --no-dedup

```

Use the deduplication API directly in Python:

```python
from skills.watch.scripts.frames import dedupe_perceptual

candidates = [
    {"path": "frame_0001.jpg", "timestamp_seconds": 0.0},
    {"path": "frame_0002.jpg", "timestamp_seconds": 0.1},
    # …

]

# Returns (surviving_candidates, number_of_dropped_frames)

survivors, dropped = dedupe_perceptual(candidates)
print(f"Kept {len(survivors)} frames, dropped {dropped}")

```

## Summary

- **Pipeline location**: Core logic resides in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) with CLI control in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py).
- **Thumbnail generation**: `_thumb_frames` uses `ffmpeg` to create 16×16 grayscale images defined by `DEDUP_THUMB = 16`.
- **Perceptual comparison**: `_frame_delta` calculates mean absolute pixel differences on a 0-255 scale with `DEDUP_THRESHOLD = 2.0`.
- **Algorithm complexity**: `_dedupe_by_deltas` runs in O(n) time using a greedy approach comparing each frame only to the last kept frame.
- **Safety**: Fail-open behavior returns original frames if thumbnail generation fails.
- **Control**: The `--no-dedup` flag disables deduplication, passing `dedup=False` to extraction engines.

## Frequently Asked Questions

### What is the time complexity of the frame deduplication algorithm?

The algorithm runs in **O(n)** linear time because `_dedupe_by_deltas` uses a greedy approach comparing each candidate only to the previously kept frame. This avoids the O(n²) cost of pairwise comparison, making it suitable for videos that generate dozens to hundreds of candidate frames.

### How does the watch skill handle deduplication failures?

The implementation **fails open**: if `_thumb_frames` encounters an `ffmpeg` error or produces mismatched byte counts, it returns an empty list, causing `dedupe_perceptual` to return the original candidate list unchanged without raising exceptions. This ensures the pipeline continues processing even when deduplication cannot run.

### Can I adjust the sensitivity of duplicate detection?

While the threshold is hardcoded as `DEDUP_THRESHOLD = 2.0` in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), you can modify this constant to make the filter more or less aggressive. Alternatively, use the `--no-dedup` flag to disable filtering entirely for maximum frame retention during debugging.

### Why does the algorithm use 16×16 thumbnails instead of full-resolution images?

Downscaling to 16×16 grayscale via `ffmpeg` reduces computational overhead while preserving enough structural information to detect true duplicates. This resolution keeps the implementation fast and memory-efficient, generating tiny byte arrays that represent the frame's visual structure without requiring heavy image processing libraries.