How the Frame Deduplication Algorithm Works in claude-video's frames.py

The frame deduplication algorithm in frames.py uses perceptual hashing (phash) to compare consecutive video frames, removing visual duplicates when the Hamming distance between their 64-bit hashes is ≤ 5 while preserving the first frame of each unique scene.

The bradautomates/claude-video repository provides a video processing pipeline that extracts frames from video sources and prepares them for downstream analysis. At the heart of this system lies skills/watch/scripts/frames.py, which implements a sophisticated frame deduplication algorithm to eliminate redundant images before transcription or computer vision tasks begin.

Core Components of the Deduplication Pipeline

The deduplication process operates in four distinct stages, each implemented as a specific function within frames.py.

Loading Frames with load_frames

The pipeline begins by reading extracted PNG or JPEG files into memory as Pillow Image objects. In skills/watch/scripts/frames.py (lines 33-44), the load_frames function handles this initialization, ensuring all frames are available for hash computation in temporal order.

Perceptual Hash Generation with hash_frame

For each loaded image, the algorithm computes a perceptual hash using the imagehash library. The hash_frame function (lines 46-55) generates a 64-bit phash that encodes the image's frequency-domain characteristics. Unlike cryptographic hashes, perceptual hashes produce similar values for visually similar images, making them tolerant to minor encoding differences or compression artifacts.

Duplicate Detection via Hamming Distance

The are_duplicates function (lines 57-64) compares consecutive frame hashes by calculating the Hamming distance between the 64-bit integers. Two frames are considered duplicates if the distance is ≤ MAX_DISTANCE (default 5). This threshold determines how visually similar frames must be before they are flagged as redundant.

Temporal Deduplication with deduplicate_frames

The core logic resides in deduplicate_frames (lines 66-84). This function iterates through the frame list and builds a new collection containing only unique visual moments:

• It always retains the first frame as the initial reference
• When the Hamming distance between the current frame and the last kept frame exceeds the threshold, the current frame is appended to the unique list and becomes the new reference
• Frames with distance ≤ threshold are discarded as duplicates

This approach preserves chronological order while ensuring that only the first frame of any static or near-static scene is retained.

Implementation Details

Here is the simplified core logic from skills/watch/scripts/frames.py demonstrating the hash-based comparison:

def deduplicate_frames(frames: List[Image.Image], max_distance: int = 5) -> List[Image.Image]:
    if not frames:
        return []

    # Compute phash for each frame

    hashes = [imagehash.phash(frame) for frame in frames]

    # Walk through hashes and drop near-duplicates

    unique = [frames[0]]          # Always keep the first frame

    last_hash = hashes[0]

    for img, h in zip(frames[1:], hashes[1:]):
        if (last_hash - h) > max_distance:   # Hamming distance > threshold → new scene

            unique.append(img)
            last_hash = h                     # Update reference hash

        # Else: duplicate – skip this frame

    return unique

The max_distance parameter controls sensitivity. Lower values (e.g., 3) create aggressive deduplication that keeps fewer frames, while higher values (e.g., 8) preserve more subtle visual changes.

Configuration and CLI Integration

The deduplication algorithm integrates with the broader video processing pipeline in watch.py. Users can configure the sensitivity via the --dedup-threshold flag, which passes the max_distance value directly to deduplicate_frames.

Practical Usage Examples

Command-Line Deduplication

To extract frames from a video and automatically deduplicate them:

python -m skills.watch.scripts.watch "https://youtu.be/abc123" --dedup

Internally, watch.py calls the deduplication function after frame extraction:

raw_frames = extract_frames(video_path)          # → list[Image]

unique_frames = deduplicate_frames(raw_frames)   # ← deduplication step

Direct Python API Usage

Import the deduplication function directly for custom workflows:

from pathlib import Path
from PIL import Image
from skills.watch.scripts.frames import deduplicate_frames

# Load previously extracted frames

frame_dir = Path("my_video_frames")
frames = [Image.open(p) for p in sorted(frame_dir.glob("*.png"))]

# Apply deduplication with strict threshold

clean_frames = deduplicate_frames(frames, max_distance=3)

print(f"Kept {len(clean_frames)} out of {len(frames)} frames")

Custom Threshold Implementation

For specialized use cases requiring different sensitivity:

from skills.watch.scripts.frames import deduplicate_frames

def strict_dedup(frames):
    # Keep frames that differ by at least 8 bits

    return deduplicate_frames(frames, max_distance=8)

# Use in your processing pipeline

unique_frames = strict_dedup(frames)

Summary

• Perceptual hashing via imagehash.phash() generates 64-bit signatures that tolerate minor visual variations while capturing scene content.
• Hamming distance compares consecutive frames; the default threshold of 5 bits filters near-identical duplicates without losing legitimate scene changes.
• Temporal preservation ensures the first frame of any duplicate sequence is always retained, maintaining chronological integrity.
• Configurable sensitivity allows tuning via the max_distance parameter or the --dedup-threshold CLI flag in watch.py.
• The algorithm processes frames in O(n) time complexity, making it efficient for long video sequences.

Frequently Asked Questions

What is perceptual hashing and why does frames.py use it?

Perceptual hashing creates a fingerprint of an image based on its visual content rather than binary data. The phash algorithm used in frames.py analyzes frequency-domain characteristics to produce a 64-bit hash where similar images produce similar hashes. This allows the frame deduplication algorithm to identify visually identical frames even if they have different file sizes or compression artifacts, unlike traditional checksums like MD5.

How does the Hamming distance threshold affect deduplication results?

The Hamming distance measures how many bits differ between two hashes. In skills/watch/scripts/frames.py, the default max_distance of 5 means frames with 5 or fewer different bits are considered duplicates. Lowering this value to 2 or 3 makes the algorithm more aggressive, keeping only frames with substantial visual changes. Increasing it to 10 or higher preserves more frames, useful for videos with subtle gradient changes or slow motion.

Can I disable frame deduplication if I need every frame?

Yes. Since deduplicate_frames is called conditionally based on the --dedup flag in watch.py, omitting this flag returns the full frame sequence. When using the Python API directly, simply skip calling deduplicate_frames and use the raw output from extract_frames or load_frames instead.

Why does the algorithm keep the first duplicate frame rather than the clearest one?

The deduplicate_frames function intentionally preserves the first frame of any duplicate run to maintain strict temporal ordering. This design ensures that timestamps remain accurate for downstream transcription and analysis tasks. The first frame typically has sufficient quality for processing, and avoiding image quality comparisons keeps the algorithm fast and deterministic without requiring additional computational overhead.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}