Understanding DEDUP_THRESHOLD in claude-video: Frame Deduplication Logic

DEDUP_THRESHOLD is a floating-point value of 2.0 defined in skills/watch/scripts/frames.py that represents the maximum mean absolute per-pixel difference (on a 0–255 grayscale scale) between two video frames, where values at or below this threshold mark frames as near-duplicates to be dropped during the watch skill's deduplication process.

The bradautomates/claude-video repository implements intelligent video processing through specialized skills, particularly the watch skill that analyzes content frame by frame. At the heart of this skill's efficiency lies the DEDUP_THRESHOLD constant, which governs how aggressively the system removes redundant visual information before sending frames to downstream LLM processing.

What Is DEDUP_THRESHOLD?

In skills/watch/scripts/frames.py, the codebase defines DEDUP_THRESHOLD as a floating-point value of 2.0. This constant establishes the upper bound for the mean absolute per-pixel difference between two down-scaled thumbnail images when determining if they represent duplicate visual content.

The threshold operates on 16×16 grayscale thumbnails generated from source video frames. When the watch skill processes a video, it compares each candidate frame against the previous retained frame. If the computed mean absolute difference across all pixels is less than or equal to 2.0, the system classifies the later frame as a near-duplicate and discards it. If the difference exceeds this value, the frame is preserved as a new reference point for subsequent comparisons.

How DEDUP_THRESHOLD Works in Frame Deduplication

The deduplication logic implemented in the claude-video source code centers on the internal _frame_delta function, which calculates the mean absolute difference between two thumbnail arrays. The process follows this sequence:

• Extract 16×16 grayscale thumbnails from candidate frames
• Compute the per-pixel absolute difference using _frame_delta
• Compare the result against DEDUP_THRESHOLD
• If delta ≤ 2.0: Drop the frame as a duplicate
• If delta > 2.0: Keep the frame and set it as the new reference

According to the repository's README, this logic ensures that "if that difference is at or below the threshold (2.0), the frame is a near-duplicate and is dropped. Otherwise it's kept and becomes the new reference."

Why 2.0? The Empirical Rationale Behind the Threshold

The value of 2.0 was selected through empirical testing across diverse video types, including static screen recordings, slide decks, talking-head presentations, and high-motion clips. This specific threshold strikes a calculated balance between aggressive deduplication and preservation of meaningful visual changes.

At 2.0 on the 0–255 grayscale scale, the threshold proves conservative enough to retain subtle but significant visual updates—such as new bullet points appearing on slides, minor cursor movements, or scrolling text—while eliminating truly identical frames that would otherwise waste tokens in downstream LLM processing. Values higher than 2.0 risked missing important slide transitions, while lower values failed to collapse redundant frames from static camera shots.

The inclusive nature of the threshold (treating exactly 2.0 as a duplicate) is explicitly verified in the test suite. The file tests/test_dedup.py contains validation logic confirming that a delta exactly equal to 2.0 triggers deduplication, ensuring consistent boundary behavior.

Working with DEDUP_THRESHOLD in Code

While the default threshold works for most use cases, the claude-video codebase provides flexibility for custom sensitivity requirements.

Using the Default Threshold

When calling extract_scene_or_uniform, the deduplication applies automatically using the built-in DEDUP_THRESHOLD:

from pathlib import Path
import frames

# Extract frames with automatic deduplication

out, meta = frames.extract_scene_or_uniform(
    "my_video.mp4",
    Path("out_dir"),
    fps=2.0,
    target_frames=50,
    max_frames=100,
)
print(f"Dropped {meta['deduped_count']} duplicate frames")

Overriding the Threshold for Custom Sensitivity

For scenarios requiring stricter or looser duplicate detection, the internal _dedupe_by_deltas helper accepts a custom threshold parameter:

survivors, dropped = frames._dedupe_by_deltas(
    candidates,               # List of frame metadata dicts

    thumbnails,               # List of 16×16 grayscale thumbnails

    threshold=1.0,            # Tighter threshold → fewer duplicates removed

)

Inspecting Computed Differences

To manually evaluate frame similarity before processing, use the _frame_delta function to retrieve the exact mean absolute difference:

delta = frames._frame_delta(thumb_a, thumb_b)
print(f"Mean per-pixel diff: {delta}")
if delta <= frames.DEDUP_THRESHOLD:
    print("Frames are considered duplicates")

Summary

• DEDUP_THRESHOLD is defined as 2.0 in skills/watch/scripts/frames.py and represents the maximum mean absolute per-pixel difference (0–255 scale) for duplicate detection.
• The threshold uses 16×16 grayscale thumbnails to compare frames efficiently without processing full-resolution images.
• A value of 2.0 was chosen empirically to balance aggressive deduplication of identical frames against preservation of meaningful visual changes like slide transitions and text updates.
• The threshold is inclusive: differences exactly equal to 2.0 trigger deduplication, as verified by tests/test_dedup.py.
• Developers can override the default threshold via the _dedupe_by_deltas helper function when processing video content with claude-video.

Frequently Asked Questions

What file defines DEDUP_THRESHOLD in claude-video?

The constant is defined in skills/watch/scripts/frames.py at line 37, where it is set to the floating-point value 2.0. This file also contains the core deduplication logic including the _frame_delta and _dedupe_by_deltas functions used by the watch skill.

How does DEDUP_THRESHOLD affect video processing costs?

By filtering out near-duplicate frames before they reach the LLM, DEDUP_THRESHOLD directly reduces the number of images processed during the watch skill execution. Since token costs scale with the number of frames analyzed, setting an appropriate threshold prevents wasting resources on visually identical content while ensuring important frames are preserved for analysis.

Can I adjust DEDUP_THRESHOLD for different video types?

Yes. While the default value of 2.0 works well for mixed content, you can pass a custom threshold parameter to frames._dedupe_by_deltas() when working with specialized content. For example, use a lower threshold (e.g., 1.0) for high-fidelity analysis where subtle changes matter, or a higher threshold for content with significant compression artifacts or noise.

Why is the threshold inclusive at exactly 2.0?

The inclusive boundary (where ≤ 2.0 equals duplicate) ensures consistent behavior at the threshold limit. This design choice is validated by the test suite in tests/test_dedup.py, which specifically verifies that frames with a delta exactly equal to 2.0 are correctly identified as duplicates and removed from the processing queue.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}