Frame Deduplication in Claude Video: A Perceptual Pipeline for Removing Duplicate Frames

Frame deduplication in Claude Video is a three-step perceptual pipeline that downsamples extracted frames to 16×16 grayscale thumbnails, calculates mean absolute pixel differences between consecutive images, and greedily drops frames with a delta ≤ 2.0 to eliminate redundant video segments.

The bradautomates/claude-video repository implements an intelligent frame deduplication system designed to reduce storage and processing overhead after video extraction. Located in skills/watch/scripts/frames.py, this system automatically filters near-identical frames whether they originate from uniform sampling, scene-change detection, or keyframe extraction engines.

How Frame Deduplication Works in Claude Video

The deduplication logic follows a computationally efficient three-stage process that balances accuracy with speed.

Step 1: Thumbnail Generation with FFmpeg

Every extracted JPEG frame is converted into a tiny grayscale thumbnail to enable fast comparison. The system uses a single ffmpeg pass to downscale images to 16 × 16 pixels (DEDUP_THUMB = 16), stripping color information to focus on structural similarity rather than chromatic variation. This transformation occurs in the _thumb_frames helper function within skills/watch/scripts/frames.py (lines 24-33).

Step 2: Per-Pixel Similarity Measurement

The _frame_delta function computes the mean absolute per-pixel difference between consecutive 16×16 thumbnails. This metric provides a normalized similarity score where lower values indicate higher visual correspondence. If the computed delta falls at or below 2.0 (DEDUP_THRESHOLD = 2.0), the frames are classified as belonging to the "same shot" and marked for removal (lines 15-22).

Step 3: Greedy Duplicate Removal and Re-indexing

The _dedupe_by_deltas function implements a greedy retention algorithm. Starting with the first frame as the reference, each subsequent thumbnail compares against the last kept frame rather than its immediate predecessor. Frames failing the threshold test are deleted from disk and excised from the frame list. Surviving frames are re-indexed sequentially from 0 to n-1 to maintain a clean numeric sequence (lines 82-88).

The dedupe_perceptual Entry Point

The public API for frame deduplication is the dedupe_perceptual function, which orchestrates the entire pipeline. This function accepts a list of frame paths, internally calls _thumb_frames to generate comparisons, invokes _dedupe_by_deltas for filtering, and returns a tuple containing (kept_frames, dropped_count).

According to the source code in skills/watch/scripts/frames.py (lines 63-71), this entry point serves as the standard interface used by both the Python API and CLI tooling after any extraction operation completes.

Configuring Deduplication Thresholds

The deduplication sensitivity is controlled by two constants defined at the module level:

• DEDUP_THUMB = 16: Sets the thumbnail resolution for comparison (16×16 pixels)
• DEDUP_THRESHOLD = 2.0: Defines the maximum mean absolute difference allowed before frames are considered distinct

Lowering the threshold increases strictness (fewer frames dropped), while raising it aggressively removes more frames. These values are hardcoded in the current implementation but govern the perceptual tolerance of the entire system.

Practical Usage Examples

Python API Implementation

Import the extraction and deduplication functions directly from the frames module:

from pathlib import Path
from skills.watch.scripts.frames import extract, dedupe_perceptual

# Extract frames at 1 FPS without immediate deduplication

frames = extract(
    video_path="lecture.mp4",
    out_dir=Path("output/frames"),
    fps=1.0,
    resolution=512,
    max_frames=100,
)

# Apply perceptual frame deduplication

unique_frames, dropped_count = dedupe_perceptual(frames)
print(f"Retained {len(unique_frames)} unique frames, removed {dropped_count} duplicates")

Command-Line Interface

The CLI automatically runs deduplication after extraction unless explicitly disabled:

# Extract and deduplicate automatically (default behavior)

python -m skills.watch.scripts.frames input.mp4 ./frames --max-frames 100

# Skip deduplication entirely

python -m skills.watch.scripts.frames input.mp4 ./frames --max-frames 100 --no-dedup

The argument parser handling these flags resides in skills/watch/scripts/frames.py (lines 160-180), ensuring the --no-dedup flag propagates correctly through the extraction flow (lines 47-50).

Integration with the Video Processing Pipeline

Frame deduplication executes automatically following any frame extraction strategy. Whether the system uses uniform sampling (fps based), scene-change detection, or native keyframe extraction, the pipeline flows through extract() → dedupe_perceptual() unless the --no-dedup flag is present. This ensures consistent redundancy removal regardless of how frames were initially harvested from the source video.

Summary

• Frame deduplication in Claude Video operates through a three-step perceptual pipeline implemented in skills/watch/scripts/frames.py.
• The system generates 16×16 grayscale thumbnails via FFmpeg to enable rapid visual comparison.
• Mean absolute pixel differences ≤ 2.0 trigger duplicate classification, with frames compared against the last retained image rather than sequential neighbors.
• The dedupe_perceptual function serves as the primary entry point, returning kept frames and drop counts after cleaning the filesystem and re-indexing survivors.
• Deduplication runs automatically after all extraction modes but can be bypassed using the --no-dedup CLI flag.

Frequently Asked Questions

What algorithm does Claude Video use for frame deduplication?

Claude Video uses a perceptual hashing approach based on downsampled thumbnails rather than cryptographic hashes. It converts frames to 16×16 grayscale images and calculates mean absolute pixel differences between consecutive frames. This method detects near-duplicate frames even if they have minor compression artifacts or lighting variations, unlike strict file-hash comparisons.

How can I disable frame deduplication when processing videos?

Add the --no-dedup flag when running the CLI command. The argument parser in skills/watch/scripts/frames.py captures this flag (lines 160-180) and passes it to the extraction logic, which conditionally skips the dedupe_perceptual() call (lines 47-50). When using the Python API, simply do not call dedupe_perceptual() after extract().

Why does Claude Video use 16×16 thumbnails for comparison?

The 16×16 resolution (DEDUP_THUMB = 16) provides sufficient granularity to detect scene changes while minimizing computational overhead. This size creates a 256-byte grayscale representation that fits easily in memory and allows rapid pixel-wise comparison via the _frame_delta function without requiring GPU acceleration or complex feature extraction.

What happens to the frame indices after deduplication?

Surviving frames are re-indexed sequentially from 0 to n-1 during the _dedupe_by_deltas process (lines 82-88). The physical files are renamed to remove gaps caused by deleted duplicates, ensuring downstream processing pipelines receive a contiguous numeric sequence without missing indices.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}