How Frame Deduplication Works in Claude's Watch Skill

The watch skill removes visually identical frames by generating 16×16 grayscale thumbnails, calculating mean pixel differences, and applying a greedy O(n) filter that drops frames below a threshold of 2.0 to compress static sequences while preserving distinct moments.

The watch skill in the bradautomates/claude-video repository extracts representative frames from video content, but raw extraction often yields redundant images from static scenes or slow transitions. Frame deduplication solves this by collapsing near-duplicate frames into single representatives, keeping the output compact without sacrificing visual diversity. This article explains the perceptual deduplication pipeline implemented in skills/watch/scripts/frames.py and controlled via the --no-dedup CLI flag.

The Three-Stage Deduplication Pipeline

After initial frame extraction, the deduplication process converts high-resolution JPEGs into comparable fingerprints and filters them chronologically using three coordinated functions.

Thumbnail Generation with FFmpeg

The _thumb_frames function in skills/watch/scripts/frames.py (lines 24-41) downsizes each candidate frame to a tiny 16×16 grayscale thumbnail using a single ffmpeg pass. This resolution, defined by the constant DEDUP_THUMB = 16, reduces computational overhead while preserving enough structural information to detect true duplicates. The implementation remains pure-stdlib compatible, processing the byte output from ffmpeg without external image libraries.

Mean-Pixel Difference Calculation

Once thumbnails exist, the _frame_delta function (lines 15-22) computes the average absolute per-pixel difference between two thumbnail byte arrays on a 0-255 scale. If the mean difference falls below DEDUP_THRESHOLD = 2.0—approximately 0.8% average change per pixel—the frames are considered near-identical and marked for removal. This conservative threshold ensures only virtually identical frames (such as duplicate frames in static screen recordings) are collapsed, while preserving fast-moving slides or scrolling terminals.

Greedy Chronological Filtering

The _dedupe_by_deltas function (lines 79-87) implements a greedy algorithm that walks the chronological candidate list in O(n) time. It keeps the first frame and compares each subsequent thumbnail only to the last kept frame. When the delta exceeds the threshold, the frame survives and becomes the new comparison baseline; otherwise, the JPEG file is deleted immediately. This approach respects visual continuity—static segments collapse to single frames while genuine changes (even subtle slide transitions) break the chain because the delta exceeds the threshold.

Public API and Integration

The dedupe_perceptual Entry Point

The dedupe_perceptual function (lines 63-71) serves as the public interface used by extraction engines throughout the skill. It guards against trivial inputs (len(candidates) <= 1), orchestrates thumbnail generation via _thumb_frames, and delegates to _dedupe_by_deltas. The function returns a tuple of (surviving_candidates, dropped_count), allowing callers to report compression statistics.

CLI Integration and Metadata

In skills/watch/scripts/watch.py (lines 212-224), the --no-dedup flag toggles the dedup boolean passed to extract_keyframes and extract_scene_or_uniform. When deduplication is enabled, it runs after frame extraction, and engines record deduped_count in result metadata. The final report displays the count of dropped frames, such as "2 near-duplicate(s) dropped."

Configuration and Threshold Selection

The DEDUP_THRESHOLD = 2.0 value is deliberately conservative. On the 0-255 pixel value scale, this threshold only collapses frames that are virtually identical, preventing the loss of meaningful content in fast-moving videos while effectively compressing redundant static sequences.

Fail-Open Safety Mechanisms

If thumbnail creation fails due to ffmpeg errors or byte count mismatches, _thumb_frames returns an empty list, causing dedupe_perceptual to return the original candidates unchanged. This ensures the extraction pipeline never crashes because of deduplication failures, maintaining availability over strict deduplication.

Code Examples

Run the watch skill with default deduplication enabled (removes near-duplicates):

watch https://example.com/video.mp4

Disable deduplication to keep every extracted frame for debugging:

watch https://example.com/video.mp4 --no-dedup

Use the deduplication API directly in Python:

from skills.watch.scripts.frames import dedupe_perceptual

candidates = [
    {"path": "frame_0001.jpg", "timestamp_seconds": 0.0},
    {"path": "frame_0002.jpg", "timestamp_seconds": 0.1},
    # …

]

# Returns (surviving_candidates, number_of_dropped_frames)

survivors, dropped = dedupe_perceptual(candidates)
print(f"Kept {len(survivors)} frames, dropped {dropped}")

Summary

• Pipeline location: Core logic resides in skills/watch/scripts/frames.py with CLI control in watch.py.
• Thumbnail generation: _thumb_frames uses ffmpeg to create 16×16 grayscale images defined by DEDUP_THUMB = 16.
• Perceptual comparison: _frame_delta calculates mean absolute pixel differences on a 0-255 scale with DEDUP_THRESHOLD = 2.0.
• Algorithm complexity: _dedupe_by_deltas runs in O(n) time using a greedy approach comparing each frame only to the last kept frame.
• Safety: Fail-open behavior returns original frames if thumbnail generation fails.
• Control: The --no-dedup flag disables deduplication, passing dedup=False to extraction engines.

Frequently Asked Questions

What is the time complexity of the frame deduplication algorithm?

The algorithm runs in O(n) linear time because _dedupe_by_deltas uses a greedy approach comparing each candidate only to the previously kept frame. This avoids the O(n²) cost of pairwise comparison, making it suitable for videos that generate dozens to hundreds of candidate frames.

How does the watch skill handle deduplication failures?

The implementation fails open: if _thumb_frames encounters an ffmpeg error or produces mismatched byte counts, it returns an empty list, causing dedupe_perceptual to return the original candidate list unchanged without raising exceptions. This ensures the pipeline continues processing even when deduplication cannot run.

Can I adjust the sensitivity of duplicate detection?

While the threshold is hardcoded as DEDUP_THRESHOLD = 2.0 in skills/watch/scripts/frames.py, you can modify this constant to make the filter more or less aggressive. Alternatively, use the --no-dedup flag to disable filtering entirely for maximum frame retention during debugging.

Why does the algorithm use 16×16 thumbnails instead of full-resolution images?

Downscaling to 16×16 grayscale via ffmpeg reduces computational overhead while preserving enough structural information to detect true duplicates. This resolution keeps the implementation fast and memory-efficient, generating tiny byte arrays that represent the frame's visual structure without requiring heavy image processing libraries.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}