How Claude Video Performs Frame Deduplication: Internal Pipeline Explained

Claude Video removes near-identical frames using a perceptual hashing pipeline that converts extracted frames to 16×16 grayscale thumbnails and drops consecutive images with a mean absolute pixel difference below 2.0.

The frame deduplication system in the bradautomates/claude-video repository prevents redundant visual data from reaching downstream processing by implementing a lightweight perceptual comparison algorithm. This process lives entirely within skills/watch/scripts/frames.py and runs automatically after every frame extraction operation unless explicitly disabled.

The Three-Step Deduplication Pipeline

The deduplication engine processes extracted JPEG frames through a sequential pipeline that minimizes disk I/O while maintaining perceptual accuracy.

Step 1: Thumbnail Generation with FFmpeg

Every extracted frame is down-scaled to a 16×16 grayscale thumbnail before comparison. The constant DEDUP_THUMB = 16 defines this resolution, and the private function _thumb_frames orchestrates a single ffmpeg pass to generate these tiny representations. This aggressive reduction removes noise and detail that would otherwise trigger false negatives during comparison, while keeping memory usage minimal.

Step 2: Perceptual Similarity Measurement

The system calculates frame similarity using the _frame_delta function, which computes the mean absolute per-pixel difference between consecutive thumbnails. If the computed delta is ≤ 2.0 (controlled by DEDUP_THRESHOLD = 2.0), the frames are classified as belonging to the "same shot" and marked for elimination. This threshold balances sensitivity—catching genuine duplicates while preserving subtle but meaningful visual changes.

Step 3: Greedy Elimination and Re-indexing

The _dedupe_by_deltas function implements a greedy algorithm that starts with the first frame and compares each subsequent thumbnail only against the last kept frame. Frames falling below the threshold are deleted from disk immediately and removed from the working list. Survivors are then re-indexed sequentially from 0 to n‑1, ensuring contiguous naming for downstream consumers.

The dedupe_perceptual Entry Point

The public interface dedupe_perceptual (lines 63‑71 in frames.py) orchestrates the complete workflow:

def dedupe_perceptual(frames: List[Path]) -> Tuple[List[Path], int]:
    """
    Returns (kept_frames, dropped_count)
    """
    thumbs = _thumb_frames(frames)
    return _dedupe_by_deltas(frames, thumbs)

This function first builds thumbnails via _thumb_frames, then delegates the actual filtering to _dedupe_by_deltas, finally returning a tuple containing the list of preserved frame paths and the count of removed duplicates.

Integration with Frame Extraction

Deduplication runs automatically after any frame extraction engine—whether uniform sampling, scene-change detection, or keyframe extraction. The CLI flow at lines 47‑50 demonstrates this integration:

frames = extract(video_path, out_dir, fps, resolution, max_frames)
if not args.no_dedup:
    frames, dropped = dedupe_perceptual(frames)

To bypass deduplication entirely, pass the --no-dedup flag when invoking the script from the command line.

Practical Usage Examples

Process a video with automatic deduplication enabled (default):

from pathlib import Path
from skills.watch.scripts.frames import extract, dedupe_perceptual

# Extract frames at 1 FPS

frames = extract(
    video_path="sample.mp4",
    out_dir=Path("output"),
    fps=1.0,
    resolution=512,
    max_frames=100
)

# Remove perceptual duplicates

unique_frames, dropped = dedupe_perceptual(frames)
print(f"Kept {len(unique_frames)} frames, dropped {dropped} duplicates")

From the command line:

python -m skills.watch.scripts.frames sample.mp4 ./output --max-frames 100

Skip deduplication when needed:

python -m skills.watch.scripts.frames sample.mp4 ./output --no-dedup

Summary

• 16×16 grayscale thumbnails provide the perceptual basis for comparison, generated via ffmpeg in _thumb_frames.
• Mean absolute difference ≤ 2.0 serves as the deduplication threshold (DEDUP_THRESHOLD), calculated by _frame_delta.
• Greedy sequential filtering in _dedupe_by_deltas compares each frame only against the last kept image, deleting matches immediately.
• Automatic execution follows every extraction unless the --no-dedup flag is provided to the CLI.
• Return signature (kept_frames, dropped_count) provides immediate feedback on compression efficiency.

Frequently Asked Questions

What threshold does Claude Video use for frame deduplication?

The system uses a mean absolute pixel difference threshold of 2.0 (defined as DEDUP_THRESHOLD = 2.0 in skills/watch/scripts/frames.py). If the average pixel difference between two 16×16 grayscale thumbnails falls at or below this value, the frames are considered duplicates and the latter is discarded.

How does the deduplication algorithm handle sequential duplicates?

The _dedupe_by_deltas function employs a greedy sliding window approach. It maintains a reference to the last kept frame and compares every subsequent frame only against that reference. If a match occurs, the current frame is deleted from disk and the reference remains unchanged. This ensures that long sequences of identical frames collapse to a single representative image while preserving the first occurrence.

Can I disable frame deduplication when processing videos?

Yes. When using the command-line interface in skills/watch/scripts/frames.py, append the --no-dedup flag (defined in the argument parser around lines 160‑180) to bypass the dedupe_perceptual call. When using the Python API, simply omit the call to dedupe_perceptual after extract().

Which source files implement the frame deduplication logic?

The core implementation resides in skills/watch/scripts/frames.py, which contains dedupe_perceptual, _thumb_frames, _frame_delta, and _dedupe_by_deltas. The pipeline orchestrator in skills/watch/scripts/watch.py invokes these functions during the full watch workflow, while tests/test_dedup.py provides unit tests against synthetic video data.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}