How the Frame Budget Is Applied After Deduplication in claude-video

In claude-video, the frame budget is enforced after deduplication by evenly sampling the surviving frames down to the requested limit, guaranteeing temporal coverage while never exceeding the configured maximum.

The claude-video project implements a sophisticated frame extraction pipeline that balances quality and efficiency by removing redundant content before applying strict output limits. Understanding exactly how the frame budget is applied after deduplication is critical for optimizing video processing workflows and predicting memory usage. This article examines the specific implementation details in the source code, walking through the perceptual deduplication stage and subsequent budget enforcement mechanisms found in skills/watch/scripts/frames.py.

The Two-Stage Pipeline: Deduplication Then Budgeting

The extraction engine processes video frames through a strict two-phase sequence. First, it eliminates visually redundant content. Second, it enforces the user-defined frame budget on the remaining unique frames.

Stage 1: Perceptual Deduplication

Before any budget constraints are applied, the pipeline collects candidate frames via uniform sampling, scene-change detection, or key-frame decoding. These candidates pass through dedupe_perceptual, which removes near-identical frames by comparing low-resolution grayscale thumbnails.

According to the implementation in skills/watch/scripts/frames.py (lines 64-73), this function returns a reduced list of survivors along with a count of dropped frames. This step ensures that subsequent budget calculations operate only on visually distinct content, preventing quota waste on redundant imagery.

Stage 2: Even Sampling to Enforce the Limit

After deduplication completes, the surviving frames are passed to _even_sample to enforce the frame budget. This helper selects n evenly-spaced frames from the input list, always preserving the first and last frames to maintain temporal boundaries. As implemented in skills/watch/scripts/frames.py (lines 93-101), the function physically deletes any JPEG files that fall outside the selected subset, returning exactly the requested count (or fewer if insufficient frames exist).

Where Budget Enforcement Happens in the Codebase

The frame budget is applied consistently across all high-level extraction routines. Each routine follows the same pattern: extract candidates, deduplicate, then cap results with _even_sample.

extract_scene_or_uniform (scene-change engine): After calling dedupe_perceptual, the survivors are capped with _even_sample to max_frames (or returned uncapped if no limit is set). This logic appears in skills/watch/scripts/frames.py at lines 44-46.

extract_keyframes (key-frame engine): Similarly, after deduplication removes duplicates from decoded key-frames, the budget is enforced via _even_sample targeting max_frames, as seen at lines 73-75 of the same file.

Uniform fallback: When the system falls back to uniform sampling, the same deduplication and budgeting sequence applies, ensuring consistent behavior regardless of extraction strategy.

Practical Implementation Examples

The following examples demonstrate the exact sequence of operations when processing video with a frame budget.

Example 1: Scene-Based Extraction with Budget

This illustrates the complete pipeline using the high-level API, which internally handles both deduplication and budget enforcement:

from pathlib import Path
from skills.watch.scripts.frames import extract_scene_or_uniform

video_path = "sample.mp4"
out_dir = Path("out")

# Extract with automatic deduplication and a hard cap of 50 frames

frames, meta = extract_scene_or_uniform(
    video_path,
    out_dir,
    fps=2.0,                # Sampling rate chosen by auto_fps()

    target_frames=50,       # Desired budget

    resolution=512,
    max_frames=50,          # Hard enforcement of frame budget

    dedup=True,             # Enable perceptual deduplication first

)

print(f"Returned {len(frames)} frames (budget applied after dedup)")
print(meta)

Example 2: Manual Pipeline Steps

For granular control, you can invoke the deduplication and budgeting steps separately:

from pathlib import Path
from skills.watch.scripts.frames import (
    extract_scene_candidates,
    dedupe_perceptual,
    _even_sample
)

video_path = "sample.mp4"
tmp_dir = Path("tmp")

# Generate initial candidates

candidates = extract_scene_candidates(video_path, tmp_dir)
print(f"Detected {len(candidates)} raw candidates")

# Step 1 – Deduplication (happens first)

deduped, dropped = dedupe_perceptual(candidates)
print(f"Dropped {dropped} near-duplicate frames")

# Step 2 – Frame budget enforcement (happens second)

budget = 30
final_frames = _even_sample(deduped, budget)
print(f"Final frame count respecting budget: {len(final_frames)}")

This explicit ordering confirms that deduplication always precedes budget application, ensuring the frame limit is calculated against unique visual content only.

Summary

• Deduplication first: The dedupe_perceptual function removes near-duplicate frames using perceptual hashing before any budget constraints are evaluated.
• Even sampling second: The _even_sample helper enforces the frame budget by selecting evenly spaced frames from the deduplicated set, always preserving temporal endpoints.
• Consistent application: Both extract_scene_or_uniform and extract_keyframes apply this two-stage pattern, ensuring predictable frame counts across all extraction modes.
• Physical cleanup: The budgeting step deletes discarded JPEG files, not just list references, managing disk space proactively.

Frequently Asked Questions

Does deduplication happen before or after the frame budget is applied?

Deduplication happens before the frame budget is applied. The pipeline first runs dedupe_perceptual to eliminate redundant frames, then passes the survivors to _even_sample to enforce the max_frames limit. This sequence ensures the budget is consumed only by visually unique content.

What algorithm does claude-video use for perceptual deduplication?

The system uses perceptual hashing of low-resolution grayscale thumbnails. The dedupe_perceptual function in skills/watch/scripts/frames.py compares these compact representations to identify near-identical frames while tolerating minor compression artifacts or encoding variations.

How does _even_sample decide which frames to keep when enforcing the budget?

_even_sample calculates evenly spaced indices across the deduplicated list, always keeping the first and last frames to preserve temporal range. It then deletes the JPEG files corresponding to any frames that fall between these indices but are not selected, physically removing them from the output directory.

Can I disable the frame budget while keeping deduplication enabled?

Yes. Setting max_frames to None or omitting the parameter in extraction calls disables the budgeting step while still allowing dedupe_perceptual to run. In this mode, all unique frames surviving deduplication are retained without the final capping step performed by _even_sample.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}