Near-Duplicate Frame Detection in Claude-Video: A Lightweight Perceptual Algorithm

Claude-Video detects near-duplicate frames by comparing 16×16 grayscale thumbnails using a mean absolute difference threshold of 2.0, greedily removing consecutive visually identical frames while preserving scene changes.

The bradautomates/claude-video repository implements a fast, deterministic near-duplicate frame detection system that runs entirely within the Python standard library. This perceptual deduplication step activates after frame extraction to collapse static shots, duplicate slides, or terminal screenshots into a single representative frame, significantly reducing storage and processing overhead without requiring heavy computer vision dependencies.

The Three-Stage Detection Pipeline

The algorithm operates in skills/watch/scripts/frames.py and consists of three distinct phases: thumbnail generation, pixel-level comparison, and greedy sequential filtering.

Stage 1: Thumbnail Generation

First, the system downscales each extracted JPEG into a tiny grayscale thumbnail. The constants DEDUP_THUMB = 16 and DEDUP_THRESHOLD = 2.0 are defined at the module level in frames.py to control this process.

The helper function _thumb_frames invokes ffmpeg once to read all JPEG files, scales them to 16×16 pixels, forces a gray pixel format, and returns the raw pixel bytes for every frame. This batch operation minimizes subprocess overhead while normalizing color information to luminance only.

# From skills/watch/scripts/frames.py

DEDUP_THUMB = 16        # Thumbnail dimension (16x16)

DEDUP_THRESHOLD = 2.0   # Mean absolute difference threshold

Stage 2: Pixel-Level Comparison

The function _frame_delta receives two thumbnail byte buffers and computes the mean absolute difference per pixel on a scale of 0–255. If the buffers differ in length—indicating mismatched extraction metadata—the function returns inf to prevent accidental deduplication of incompatible frames.

This metric avoids expensive perceptual hashing algorithms while reliably detecting visually identical content. The calculation is straightforward arithmetic on raw bytes, ensuring predictable performance regardless of input complexity.

Stage 3: Greedy Sequential Filtering

The dedupe_perceptual function orchestrates the deduplication by calling _dedupe_by_deltas. The algorithm walks the chronological frame list and maintains a reference to the last kept thumbnail:

1. Keep the first frame as the initial reference.
2. For each subsequent frame, compare its thumbnail to the last kept thumbnail.
3. If the mean absolute difference is ≤ DEDUP_THRESHOLD (2.0), delete the JPEG file and omit the frame from the results.
4. If the difference exceeds the threshold, keep the frame and update the reference.

This greedy approach guarantees that only consecutive near-identical frames are merged, preserving rapid cuts and motion while eliminating static redundancies. After the pass completes, survivors are re-indexed and the function returns the count of dropped frames.

Configuration and CLI Integration

The deduplication logic is tightly integrated with the extraction engines. The extract, extract_scene_or_uniform, and extract_keyframes functions all invoke dedupe_perceptual automatically unless the user supplies the --no-dedup flag.

In skills/watch/scripts/watch.py, the CLI parser exposes this toggle:

# From skills/watch/scripts/watch.py

parser.add_argument('--no-dedup', action='store_true',
                    help='Disable perceptual deduplication')

When enabled (the default), the pipeline ensures that the final frame set remains compact yet visually representative.

Practical Usage Examples

Programmatic Deduplication

You can invoke the deduplication logic directly in custom scripts after extracting frames:

from pathlib import Path
from skills.watch.scripts.frames import dedupe_perceptual

# candidates is a list of frame dicts from any extraction function

candidates = [{'path': 'frame_0001.jpg', 'timestamp': 0.0}, 
              {'path': 'frame_0002.jpg', 'timestamp': 0.1}, ...]

unique_frames, dropped = dedupe_perceptual(candidates)

print(f"Kept {len(unique_frames)} frames, removed {dropped} near-duplicates")

Command-Line Usage

Run the default extraction with automatic deduplication:

python3 -m skills.watch.scripts.watch video.mp4 out-dir

To preserve every raw frame without deduplication:

python3 -m skills.watch.scripts.watch video.mp4 out-dir --no-dedup

Summary

• Thumbnail-based comparison: The algorithm reduces frames to 16×16 grayscale thumbnails using ffmpeg, minimizing memory and computational costs.
• Mean absolute difference: A threshold of 2.0 (on a 0–255 scale) determines visual similarity without requiring complex perceptual hashing libraries.
• Greedy sequential filtering: The algorithm compares each frame only to the last kept frame, ensuring that rapid scene changes are preserved while consecutive duplicates are removed.
• Integrated pipeline: Deduplication runs automatically after extraction in skills/watch/scripts/frames.py unless disabled via the --no-dedup flag in skills/watch/scripts/watch.py.

Frequently Asked Questions

What threshold does Claude-Video use for near-duplicate frame detection?

Claude-Video uses a mean absolute difference threshold of 2.0 as defined by the DEDUP_THRESHOLD constant in skills/watch/scripts/frames.py. This value represents the average per-pixel difference on a 0–255 grayscale scale, effectively collapsing frames that differ only by compression artifacts or minor noise.

How does the algorithm handle rapid scene changes?

The greedy sequential comparison in _dedupe_by_deltas compares each frame only to the last kept frame, not to all previous frames. This design ensures that rapid cuts—where consecutive frames differ significantly—are preserved in the output, while only sequences of visually identical frames are collapsed.

Can I disable near-duplicate frame detection?

Yes. Pass the --no-dedup flag when running the CLI via skills/watch/scripts/watch.py. This flag is exposed in the argument parser and respected by all extraction engines (extract, extract_scene_or_uniform, extract_keyframes), causing the pipeline to skip the dedupe_perceptual call entirely.

Why use mean absolute difference instead of perceptual hashing?

The implementation prioritizes speed and zero dependencies. Mean absolute difference on 16×16 thumbnails requires only standard library arithmetic operations, whereas perceptual hashing libraries introduce additional dependencies and computational overhead. This approach reliably detects static shots and duplicate slides while maintaining deterministic, fast performance across platforms.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}