How Frame Deduplication Works in watch.skill: Perceptual Hashing Explained

Frame deduplication in watch.skill eliminates visually redundant video frames by computing perceptual hashes (phash) for each image and removing duplicates based on Hamming distance thresholds.

The watch skill in the bradautomates/claude-video repository processes video files to extract a minimal, information-rich set of frames for downstream transcription and analysis. Rather than processing every single frame, the skill implements an intelligent deduplication pipeline in skills/watch/scripts/frames.py that uses perceptual hashing to detect and discard near-identical images while preserving visual changes.

The Deduplication Pipeline in skills/watch/scripts/frames.py

The deduplication workflow operates in four distinct stages, implemented within the frame extraction logic.

Frame Extraction with ffmpeg

The pipeline begins by invoking ffmpeg to extract frames from the input video. Depending on configuration, the skill either captures every frame or samples at an auto-adjusted FPS rate, writing the output as PNG files to a temporary directory. This raw extraction ensures all visual data is available for the subsequent hashing phase.

Perceptual Hash Generation

Each extracted PNG frame is opened using Pillow and processed through the imagehash library to generate a phash (perceptual hash). Unlike cryptographic hashes, perceptual hashes capture the visual structure of the image and remain stable across minor compression artifacts or encoding variations. This hash computation is the foundation for similarity detection.

Hamming Distance Comparison

As frames are processed sequentially, the current frame's hash is compared against the previously kept frame's hash using Hamming distance. The Hamming distance measures the number of differing bits between the two hash values, providing a numeric representation of visual similarity. Lower values indicate nearly identical images.

Threshold-Based Filtering

The comparison uses a configurable DEDUP_THRESHOLD parameter, defined in the skill's configuration. When the Hamming distance between the current and reference hash is less than or equal to DEDUP_THRESHOLD (defaulting to 0 for exact matches), the current frame is considered a duplicate and deleted from the output directory. Otherwise, the frame is retained and becomes the new reference hash for subsequent comparisons.

Configuration and Default Thresholds

The deduplication behavior is controlled through constants typically defined in skills/watch/scripts/config.py. The default DEDUP_THRESHOLD of 0 enforces strict deduplication, requiring perceptual hashes to match exactly before frames are considered duplicates. Increasing this threshold allows the system to remove frames with subtle visual differences, further reducing the dataset size at the cost of potentially losing minor visual details.

Practical Implementation Examples

To extract frames with automatic deduplication, the skill provides a high-level interface:

from skills.watch.scripts.frames import extract_and_dedup_frames

video_path = "/tmp/video.mp4"
output_dir = "/tmp/frames"

# Runs ffmpeg extraction then removes perceptual duplicates

extract_and_dedup_frames(
    video_path,
    output_dir,
    dedup_threshold=0,   # 0 = exact perceptual hash match required

)

For custom workflows, the hash-based deduplication logic can be implemented manually:

from PIL import Image
import imagehash
import pathlib

def dedup_images(image_dir: pathlib.Path, threshold: int = 0):
    """
    Remove duplicate frames based on perceptual hash similarity.
    """
    kept = []
    prev_hash = None
    
    for img_path in sorted(image_dir.glob("*.png")):
        cur_hash = imagehash.phash(Image.open(img_path))
        
        if prev_hash is None or (prev_hash - cur_hash) > threshold:
            kept.append(img_path)
            prev_hash = cur_hash
        else:
            img_path.unlink()  # Delete duplicate frame

            
    return kept

# Example usage

dedup_images(pathlib.Path("/tmp/frames"), threshold=0)

Testing the Deduplication Logic

The frame deduplication implementation is validated in tests/test_dedup.py. This test suite creates synthetic video sequences with known duplicate patterns, executes the extraction pipeline, and asserts that the Hamming distance filtering correctly eliminates redundant frames according to the specified threshold. These tests ensure that the perceptual hashing integration remains stable across updates to the imagehash library or ffmpeg parameters.

Integration with the Watch Skill Pipeline

The deduplication system functions as part of a larger processing chain. First, skills/watch/scripts/download.py retrieves the target video source. Then, skills/watch/scripts/frames.py handles the extraction and filtering. By removing duplicates before transcription or analysis, the skill significantly reduces compute costs and API token usage while maintaining the full semantic content of the video through its unique visual moments.

Summary

• Perceptual hashing via imagehash.phash() converts each frame into a compact, visual-structure-aware fingerprint.
• Hamming distance comparisons in skills/watch/scripts/frames.py quantify similarity between consecutive frames.
• Configurable thresholds (DEDUP_THRESHOLD) control strictness, defaulting to 0 for exact matches.
• Automatic cleanup deletes duplicate PNGs from the output directory, leaving only visually distinct frames.
• Test coverage in tests/test_dedup.py validates the deduplication accuracy against synthetic video data.

Frequently Asked Questions

What algorithm does watch.skill use for perceptual hashing?

The skill uses the phash (perceptual hash) algorithm provided by the imagehash Python library. This algorithm resizes the image, converts it to grayscale, and computes a hash based on the Discrete Cosine Transform (DCT), making it robust against minor compression artifacts and brightness changes.

How does the DEDUP_THRESHOLD parameter affect frame retention?

The DEDUP_THRESHOLD defines the maximum allowable Hamming distance between two perceptual hashes for frames to be considered duplicates. A threshold of 0 (the default) requires identical hashes, while higher values allow the removal of frames with subtle visual differences. The comparison (prev_hash - cur_hash) > threshold determines whether a frame is kept or deleted.

Why does the skill delete duplicate frames rather than marking them?

The pipeline in skills/watch/scripts/frames.py physically removes duplicate PNG files using path.unlink() after extraction. This reduces storage requirements and ensures that downstream processors only handle unique visual data, optimizing both disk usage and processing time for transcription workflows.

Where is the frame deduplication logic tested?

The deduplication behavior is verified in tests/test_dedup.py, which creates controlled video sequences with known frame patterns and validates that the Hamming distance filtering and threshold logic correctly identify and remove duplicates while preserving visually distinct frames.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}