Frame Delta Threshold of 2.0 in Claude-Video: Why This Value Controls Video Frame Deduplication

The frame delta threshold in claude-video is set to 2.0 to conservatively distinguish near-identical frames (static slides, fades, terminal scrolling) from meaningful scene changes, keeping token usage low without sacrificing content fidelity.

The claude-video repository implements intelligent frame deduplication to minimize redundant visual data before processing. This article examines why the specific threshold of 2.0 was chosen for the _frame_delta comparison and how it balances aggressive duplicate removal with content preservation.

How Frame Deduplication Works in Claude-Video

Frame deduplication in claude-video operates on low-resolution thumbnails rather than full-resolution frames. This design choice prioritizes computational efficiency while maintaining sufficient accuracy for visual similarity detection.

Thumbnail Generation

Each extracted frame is converted to a 16 × 16 grayscale thumbnail before comparison. In skills/watch/scripts/frames.py, the constant DEDUP_THUMB = 16 defines this dimension:

# From skills/watch/scripts/frames.py

DEDUP_THUMB = 16  # Thumbnail size for deduplication comparison

DEDUP_THRESHOLD = 2.0  # Maximum mean pixel difference to consider duplicate

These thumbnails reduce each frame to 256 grayscale values (16 × 16), dramatically shrinking the comparison space while preserving essential luminance patterns.

The _frame_delta Comparison

The core comparison logic resides in _frame_delta, which computes the mean absolute per-pixel difference between two consecutive thumbnails. This metric measures average luminance variation across the entire thumbnail.

The deduplication pipeline then applies _dedupe_by_deltas (line 94 in frames.py) with this logic:

# Conceptual implementation based on frames.py

def _dedupe_by_deltas(candidates, thumbs):
    survivors = [candidates[0]]
    last_kept_thumb = thumbs[0]
    dropped = 0
    
    for candidate, thumb in zip(candidates[1:], thumbs[1:]):
        delta = _frame_delta(thumb, last_kept_thumb)
        if delta <= DEDUP_THRESHOLD:  # 2.0 threshold, inclusive

            dropped += 1  # Duplicate detected, skip this frame

        else:
            survivors.append(candidate)
            last_kept_thumb = thumb  # Compare against last KEPT, not last seen

    
    return survivors, dropped

Why 2.0? The Rationale Behind the Frame Delta Threshold

The 2.0 value was selected through empirical testing to address three distinct scenarios:

Near-Identical Visual Content

Static slides, slow fades, and terminal scrolling produce minimal luma variation. On a 0–255 grayscale scale, a mean difference of ≤2.0 reliably captures these cases as redundant. This conservatively low threshold ensures truly duplicate content is eliminated before token-intensive processing.

Distinct Scene Changes

Meaningful visual transitions—scene cuts, color shifts, camera movements—generate substantially larger per-pixel deltas. The 2.0 threshold sits well below typical values for these changes, ensuring distinct frames survive deduplication.

The test suite validates this behavior in tests/test_dedup.py:

• test_dedupe_keeps_all_distinct — verifies visibly different frames are preserved
• test_dedupe_compares_against_last_kept_not_previous — confirms comparison against retained frames, not merely consecutive ones

Edge Case Handling with Inclusive Comparison

The threshold uses inclusive comparison (<= rather than <). This means a delta exactly equal to 2.0 triggers duplicate removal. The test test_dedupe_threshold_is_inclusive explicitly validates this boundary behavior, ensuring consistent handling of borderline cases.

Practical Usage: Applying the Threshold

You can observe the deduplication behavior directly:

from pathlib import Path
import frames

# Assume candidates is a list of dicts from frames.extract(...)

candidates = frames.extract(video_path="/path/to/video.mp4")

# Generate thumbnails for comparison

thumbs = frames._thumb_frames([Path(c["path"]) for c in candidates])

# Apply default 2.0 threshold

survivors, dropped = frames._dedupe_by_deltas(candidates, thumbs)

print(f"Kept {len(survivors)} frames, removed {dropped} duplicates")

The DEDUP_THRESHOLD is hardcoded at 2.0 in the source. For research or debugging, you could temporarily modify it, though the default has been validated across diverse video types.

Performance Impact of the 2.0 Threshold

The threshold directly affects token efficiency and processing cost:

Threshold Behavior	Result
Too high (>5.0)	Near-duplicates retained, excessive tokens consumed
2.0 (current)	Optimal balance: redundant frames removed, content preserved
Too low (<1.0)	Subtle but meaningful changes lost, content degraded

The 2.0 value emerged from testing across presentation videos, screen recordings, and dynamic footage—domains where claude-video is commonly deployed.

Key Source Files for Frame Delta Deduplication

File	Purpose
skills/watch/scripts/frames.py	Core implementation: DEDUP_THRESHOLD = 2.0, _frame_delta(), _dedupe_by_deltas()
tests/test_dedup.py	Unit tests for threshold behavior, inclusivity, and comparison logic
tests/test_frames.py	Integration tests validating full extraction pipeline with deduplication

Summary

• Frame delta threshold of 2.0 in claude-video operates on 16×16 grayscale thumbnails to detect near-identical frames
• Mean absolute per-pixel difference ≤2.0 (on 0–255 scale) triggers duplicate removal
• Conservative value eliminates static content while preserving scene cuts and meaningful changes
• Inclusive comparison (<=) ensures boundary cases are handled as duplicates
• Token optimization is the primary goal, validated by tests/test_dedup.py

Frequently Asked Questions

What units is the frame delta threshold measured in?

The threshold represents mean absolute pixel difference on a 0–255 grayscale scale. A value of 2.0 means the average luminance difference across all 256 thumbnail pixels is at most 2 intensity levels—extremely subtle variation invisible to human perception.

Can I adjust the frame delta threshold in claude-video?

DEDUP_THRESHOLD = 2.0 is hardcoded in skills/watch/scripts/frames.py. Modifying it requires editing the source constant. The developers selected 2.0 based on cross-domain testing; alternative values risk either excessive token usage or content loss.

Why compare thumbnails instead of full-resolution frames?

Computational efficiency. A 1920×1080 frame contains 2 million pixels; a 16×16 thumbnail contains 256. The thumbnail preserves sufficient luminance structure for duplicate detection while reducing comparison cost by 99.99%.

How does claude-video handle gradual transitions like fades?

Fades and slow transitions often produce deltas below 2.0 throughout their duration. The deduplicator keeps only the first frame of such sequences, then resumes capturing once the delta exceeds threshold—effectively sampling the transition rather than storing every near-identical intermediate frame.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}