# What Is DEDUP_THRESHOLD (2.0) in Claude Video and How Is Mean Absolute Difference Calculated?

> Understand Claude Video's DEDUP_THRESHOLD 2.0 and learn how to calculate mean absolute difference to identify and remove duplicate frames from your videos.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: internals
- Published: 2026-08-02

---

**The `DEDUP_THRESHOLD` constant (2.0) sets the maximum mean-absolute per-pixel difference allowed before a frame is flagged as a near-duplicate and removed in Claude Video's deduplication pipeline.**

Claude Video (`bradautomates/claude-video`) uses perceptual deduplication to shrink video frame sets without sacrificing visual diversity. At the heart of this system lies a simple but effective comparison: tiny 16×16 grayscale thumbnails compared via mean-absolute difference, with a conservative threshold of 2.0 to catch only truly similar frames.

## How DEDUP_THRESHOLD (2.0) Controls Duplicate Detection

The **mean-absolute difference** measures average pixel intensity variation between two frames. The **`DEDUP_THRESHOLD`** of **2.0** means only frames differing by ≤2 intensity units (on a 0-255 scale) are considered duplicates. This equals roughly **0.8% maximum variation**—a deliberately tight bound ensuring near-identical frames are caught while preserving meaningful visual changes.

Frames exceeding this threshold are kept; those at or below it are dropped. This conservative approach prevents over-aggressive pruning that could strip contextually important frames from the video analysis.

## Mean Absolute Difference Calculation in _frame_delta

The core computation lives in **`_frame_delta`** within [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) (lines 38+):

```python
def _frame_delta(a: bytes, b: bytes) -> float:
    """Mean absolute per-pixel difference (0-255) between two grayscale thumbnails."""
    if not a or len(a) != len(b):
        return float("inf")                     # mismatched size ⇒ treat as different

    return sum(abs(x - y) for x, y in zip(a, b)) / len(a)

```

The function operates on raw bytes from **`DEDUP_THUMB × DEDUP_THUMB`** thumbnails—16×16 pixels, hence 256 bytes each. Here's the step-by-step breakdown:

- **Input validation** – Mismatched byte sequences return infinity, forcing different-frame treatment
- **Per-pixel absolute difference** – `abs(x - y)` computes intensity gap for each corresponding pixel pair
- **Mean calculation** – Sum of all 256 differences divided by 256 yields the final score
- **Threshold comparison** – In `_dedupe_by_deltas`, frames pass when `_frame_delta(thumb, last) > DEDUP_THRESHOLD`

## Practical Code Examples

### Compute Mean Absolute Difference Directly

```python
from pathlib import Path
from skills.watch.scripts.frames import _thumb_frames, _frame_delta

# Load two sample thumbnails (already downscaled to 16×16 gray)

thumbs = _thumb_frames([Path("frame001.jpg"), Path("frame002.jpg")])
diff = _frame_delta(thumbs[0], thumbs[1])
print(f"Mean-absolute difference: {diff:.2f}")   # ≈ 1.3 etc.

```

### Run Full Deduplication on Extracted Frames

```python
from skills.watch.scripts.frames import dedupe_perceptual

# `candidates` is a list of dicts, each containing a `"path"` to a JPEG frame.

# The function returns the filtered list and the number of frames dropped.

filtered_frames, dropped = dedupe_perceptual(candidates)   # uses DEDUP_THRESHOLD = 2.0

print(f"Dropped {dropped} near-duplicate frames")

```

## Key Source Files and Functions

| File | Purpose | Key Elements |
|------|---------|--------------|
| [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) | Deduplication engine | `DEDUP_THRESHOLD`, `_frame_delta()`, `_dedupe_by_deltas()`, thumbnail generation |
| [`tests/test_dedup.py`](https://github.com/bradautomates/claude-video/blob/main/tests/test_dedup.py) | Unit test coverage | Threshold boundary tests, mean-absolute-difference validation |
| [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) | CLI integration | `--no-dedup` flag, dropped-frame reporting |

The deduplication pipeline called from [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) (lines 64-69) respects the `--no-dedup` toggle and surfaces statistics about frame reduction.

## Why 2.0? Design Rationale for the Threshold Value

A **`DEDUP_THRESHOLD`** of **2.0** reflects careful calibration for video analysis use cases:

- **Too high (>10.0)** – Would collapse visually distinct frames, losing scene transitions
- **Too low (<1.0)** – Would retain encoding artifacts and near-identical consecutive frames
- **2.0 sweet spot** – Catches compression variations, slight camera shake, and duplicate encodings while preserving genuine content shifts

According to the `claude-video` source code, this value emerged from testing across diverse video types—screen recordings, camera footage, and mixed-content sources.

## Summary

- **`DEDUP_THRESHOLD` (2.0)** caps mean-absolute difference at 2 intensity units per pixel for duplicate classification
- **`_frame_delta`** in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) computes the metric using 16×16 grayscale thumbnails
- The threshold equals **~0.8% maximum pixel variation**—a conservative filter preserving visual diversity
- Raw byte comparison via `sum(abs(x-y))/len(a)` provides fast, allocation-light differencing
- Pipeline integration spans [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py) deduplication logic, [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) CLI handling, and [`test_dedup.py`](https://github.com/bradautomates/claude-video/blob/main/test_dedup.py) verification

## Frequently Asked Questions

### What happens if I want to disable deduplication entirely?

Pass the **`--no-dedup`** flag to the Claude Video CLI. The [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) entry point reads this flag and bypasses `dedupe_perceptual()`, returning all extracted frames unfiltered. This preserves every frame for analysis pipelines requiring complete temporal coverage.

### Can I adjust DEDUP_THRESHOLD for different video types?

The constant is hardcoded in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py). Modifying it requires editing source—there is no runtime configuration. Lower values (1.0) tighten duplicate detection for static content like slides; higher values (3.0-5.0) relax filtering for noisy footage. Rebuild and reinstall the package after changes.

### How does mean absolute difference compare to perceptual hashing?

Mean-absolute difference used by Claude Video is **computationally cheaper** than perceptual hashing (pHash, dHash) but **less robust to transformations**. It catches pixel-level duplicates efficiently but misses rescaled or rotated versions. For the video frame extraction use case—where consecutive frames share scale and orientation—this tradeoff favors speed and simplicity.

### Why use 16×16 thumbnails instead of full-resolution frames?

**Performance**: 256 pixels versus potentially millions reduces memory pressure and comparison time by 3-4 orders of magnitude. **Sufficiency**: For duplicate detection, structural similarity at thumbnail scale correlates strongly with full-frame similarity. The `DEDUP_THUMB` constant (16) balances discrimination power against computational cost.