# How Claude Video's Frame Deduplication Algorithm Works: Implementation Guide

> Learn how Claude Video's frame deduplication algorithm works. Discover its efficient implementation for removing near-duplicate frames using thumbnails and difference calculations.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: internals
- Published: 2026-07-29

---

**Claude Video removes near-duplicate frames before applying the frame-budget cap by generating 16×16 greyscale thumbnails, computing mean absolute differences, and greedily filtering against the last kept frame using a configurable threshold.**

The **Claude Video frame deduplication algorithm** is a lightweight perceptual filter implemented in the `bradautomates/claude-video` repository that eliminates redundant visual data before sending frames to Claude. This preprocessing step ensures that static screen recordings, paused video segments, and slide decks do not waste the user-specified frame budget on identical content. By comparing downscaled thumbnails rather than full-resolution images, the system trades minimal CPU overhead for significant cost savings and improved relevance in downstream AI analysis.

## The Perceptual Deduplication Pipeline

The deduplication pass operates as a three-stage pipeline that processes extracted frames before the final budget cap is applied.

### Generating 16×16 Greyscale Thumbnails

Every extracted frame is downscaled to a square of `DEDUP_THUMB` pixels (default **16 × 16**) and converted to 8-bit greyscale. This aggressive reduction preserves the visual "shape" of the image while drastically reducing the data that must be compared. In [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), the `thumb_image` helper implements this transformation around **line 440**, creating a compact representation suitable for fast pixel-wise comparison.

### Computing Mean Absolute Difference

For each candidate frame, the algorithm calculates the **mean absolute difference** between its thumbnail and the thumbnail of the *last kept* frame. This scalar distance metric reflects the visual similarity between two frames, where a value of zero indicates identical pixel data. The calculation compares corresponding pixels in the two 16×16 greyscale arrays, summing the absolute differences and dividing by the total pixel count.

### Greedy Last-Kept Comparison Logic

The frame survives the filter **only if** its distance exceeds a configurable **threshold** (default **2.0**). Because the comparison is always against the *last kept* frame rather than the immediately preceding frame, the algorithm collapses entire runs of identical or near-identical content into a single representative image. This greedy approach is implemented in the private function **`_dedupe_by_deltas`** defined at **line 479** of [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), which returns a tuple containing the list of surviving frame timestamps and the count of dropped frames.

## Source Code Implementation Details

The deduplication logic is tightly integrated into the frame extraction pipeline while remaining bypassable for specialized use cases.

### Core Dedup Logic in frames.py

The `dedupe_perceptual` function (exposed through `extract_frames`) orchestrates the thumbnail generation and filtering process. The critical private helper **`_dedupe_by_deltas`** performs the actual greedy filtering loop, maintaining a reference to the last accepted thumbnail and comparing each subsequent candidate against it. When the mean absolute difference falls below the threshold, the candidate is discarded and the drop counter increments.

### CLI Integration and Metadata Tracking

The entry point in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) exposes deduplication control through the **`--no-dedup`** flag. Lines **212** and **224** handle this argument, conditionally skipping the `dedupe_perceptual` call when the user requires every sampled frame. After processing, the system records the effectiveness of the filter in the output metadata; around **line 549**, the `deduped_count` field is populated to indicate how many redundant frames were removed from the stream.

### Budget Application Order

Deduplication occurs **before** the `--max-frames` budget cap is enforced. This sequencing ensures that the frame quota is spent exclusively on **distinct visual content** rather than wasted on redundant data. For video containing long static segments, this ordering can reduce API costs and improve context window utilization by ensuring Claude receives only visually unique information.

## CLI and Programmatic Usage

You can control the deduplication behavior through command-line arguments or by invoking the internal API directly.

### Enabling Default Deduplication

Run the watch skill with default settings to automatically remove near-duplicates:

```bash
python -m skills.watch.scripts.watch https://www.youtube.com/watch?v=example

```

### Disabling Deduplication for Motion Analysis

Pass the **`--no-dedup`** flag when processing video requiring frame-perfect granularity, such as fine-grained motion analysis or high-speed recordings where every micro-change matters:

```bash
python -m skills.watch.scripts.watch https://www.youtube.com/watch?v=example --no-dedup

```

### Direct API Access

For custom pipelines, import the deduplication routine directly from [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py):

```python
from skills.watch.scripts import frames

# candidates: list of (timestamp, path) tuples from the extractor

# thumbs: list of 16×16 greyscale thumbnails corresponding to candidates

survivors, dropped = frames._dedupe_by_deltas(candidates, thumbs, threshold=2.0)

print(f"Kept {len(survivors)} frames, dropped {dropped} near-duplicates")

```

## Frame Budget Optimization

The deduplication algorithm directly impacts cost efficiency by ensuring the `--max-frames` limit applies only to perceptually unique content. In [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py), the pipeline architecture guarantees that the `dedupe_perceptual` step completes before the budget enforcement logic executes. This design is particularly effective for screen recordings containing static UI elements, presentation slides with minimal animation, or paused video segments, where the algorithm commonly achieves compression ratios exceeding 10:1 without loss of semantic information.

## Summary

- **Thumbnail preprocessing**: All frames are downscaled to 16×16 greyscale images in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) to enable fast comparison.
- **Distance metric**: The algorithm uses mean absolute difference between thumbnails, with a default threshold of 2.0 determining deduplication boundaries.
- **Greedy filtering**: The `_dedupe_by_deltas` function implements last-kept comparison logic at line 479, collapsing runs of duplicates into single representative frames.
- **CLI control**: The `--no-dedup` flag in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) (lines 212 and 224) allows users to preserve every extracted frame when necessary.
- **Budget efficiency**: Deduplication occurs before the `--max-frames` cap, ensuring Claude receives only distinct visual content and maximizing the value of each API call.

## Frequently Asked Questions

### What is the default threshold for frame deduplication in Claude Video?

The default threshold is **2.0**, representing the mean absolute difference between 16×16 greyscale thumbnails. Frames with a distance equal to or less than this value are considered near-duplicates and discarded, while frames exceeding this threshold are kept as visually distinct content.

### How does the algorithm handle gradual transitions or slow motion?

Because the algorithm compares each frame against the *last kept* frame rather than the immediate predecessor, gradual transitions typically survive the filter until the cumulative visual change exceeds the threshold. Slow motion video will drop frames that are perceptually identical within the 16×16 greyscale space but retain frames where the mean absolute difference crosses the 2.0 boundary, preserving the temporal progression without redundant samples.

### Can I adjust the thumbnail size for deduplication?

The thumbnail size is controlled by the `DEDUP_THUMB` constant in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), which defaults to 16 pixels. While the source code allows modification of this value, changing it requires editing the constant definition and affects the granularity of the mean absolute difference calculation—larger thumbnails increase CPU overhead but may catch subtle visual differences that 16×16 sampling misses.

### Does disabling deduplication affect the frame budget calculation?

Disabling deduplication with `--no-dedup` changes which frames are eligible for the budget cap but does not change the cap itself. When deduplication is active, the `--max-frames` limit applies only to the surviving unique frames; when disabled, the limit applies to every sampled frame extracted from the video, potentially including hundreds of identical or near-identical images that would otherwise be filtered out.