How to Optimize Token Usage and Manage Frame Counts in Claude-Video

Claude-Video controls token consumption through configurable detail levels, auto-FPS calculation, scene-aware extraction, and perceptual deduplication, allowing you to balance visual context against API costs.

Claude-Video extracts frames from video files and sends them to Claude as image tokens, where each frame directly impacts your token budget. According to the bradautomates/claude-video source code, the library provides several configuration levers—from preset detail modes to custom frame caps—that let you optimize token usage while preserving essential visual context.

Set Your Frame Budget with Detail Levels

The --detail flag (or WATCH_DETAIL environment variable) selects a predefined frame budget that determines how many frames Claude-Video extracts:

• efficient: Caps at 50 frames using key-frame extraction only [watch.py L4-L5]
• balanced: Caps at 100 frames using scene-aware detection [watch.py L15-L20]
• token-burner: No cap—extracts all detected shots [watch.py L22-L24]
• transcript: No frames—returns only captions

This mapping is defined in skills/watch/scripts/config.py where the frame_cap function returns the integer limit or None for unlimited modes:

def frame_cap(detail: str) -> int | None:  # [[config.py L65-L74](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py#L65-L74)]

    if detail == "efficient":
        return 50
    if detail == "balanced":
        return 100
    return None  # token-burner & transcript have no hard cap

Calculate Optimal Frame Density with Auto-FPS

Rather than using a fixed sampling rate, Claude-Video computes an auto-FPS to meet your frame budget while respecting a hard ceiling of 2 FPS (MAX_FPS). The calculation lives in skills/watch/scripts/frames.py.

For full videos, the auto_fps function determines the sampling rate based on duration and max_frames [frames.py L22-L38]. For focused ranges (using --start/--end), the auto_fps_focus function supplies a denser budget [frames.py L41-L59].

Both paths use the _clamp_fps helper to enforce limits:

def _clamp_fps(fps: float, duration_seconds: float, max_frames: int) -> tuple[float, int]:
    fps = min(fps, MAX_FPS)                    # never exceed 2 fps

    target = min(max_frames, max(1, int(round(fps * duration_seconds))))
    return fps, target

Extract Maximum Value with Scene-Aware Sampling

When videos contain sufficient scene cuts (SCENE_MIN_FRAMES = 8), Claude-Video switches to extract_scene_or_uniform in skills/watch/scripts/frames.py [frames.py L110-L132]. This engine:

1. Extracts the first frame and ffmpeg-detected scene changes
2. Deduplicates near-identical shots
3. Even-samples down to the frame cap

For static videos with fewer than 8 cuts, it falls back to uniform sampling using the auto-FPS budget.

Reduce Token Waste with Perceptual Deduplication

Consecutive frames in slide decks or static shots often look identical. The dedupe_perceptual function in skills/watch/scripts/frames.py creates 16×16 grayscale thumbnails and drops frames whose mean-pixel difference is ≤ DEDUP_THRESHOLD = 2.0 [frames.py L64-L71].

Disable this with --no-dedup when analyzing code diffs or other content where every frame matters.

Override Defaults for Fine-Grained Control

Beyond detail levels, skills/watch/scripts/watch.py accepts several CLI arguments to directly manipulate the token budget [watch.py L30-L38]:

• --max-frames N: Overrides the frame cap derived from detail levels
• --fps F: Forces specific FPS (still clamped to MAX_FPS = 2)
• --resolution W: Reduces JPEG width (default 512px) to lower per-frame token costs
• --timestamps "10,00:30": Pins specific frames, reserved against the cap
• --no-dedup: Disables perceptual deduplication

Handle Long Videos and Token Warnings

For videos exceeding 10 minutes, the default balanced detail (100 frames) may provide sparse coverage. The CLI warns you and suggests narrowing the focus with --start/--end or using --detail token-burner [watch.py L26-L33].

If you exceed 250 frames in token-burner mode, Claude-Video warns that token costs may be high and recommends lowering resolution or adding a cap via --max-frames [watch.py L19-L25].

Practical Examples

# Low-token mode: keyframes only, capped at 50

watch https://example.com/video.mp4 --detail efficient

# Pin critical timestamps while maintaining budget

watch https://example.com/presentation.mp4 \
    --detail balanced \
    --timestamps "00:15,00:45,01:30"

# Reduce per-frame token cost

watch https://example.com/video.mp4 --resolution 256

# High-detail analysis (use with caution)

watch https://example.com/lecture.mov --detail token-burner --max-frames 200

Summary

• Detail levels provide preset frame caps (50, 100, or unlimited) based on your token budget and are defined in skills/watch/scripts/config.py
• Auto-FPS calculation dynamically adjusts sampling rates up to a 2 FPS maximum to hit frame targets without waste, implemented in skills/watch/scripts/frames.py
• Scene-aware extraction prioritizes visually distinct frames over uniform sampling when video content varies, triggered when scene cuts exceed 8 frames
• Perceptual deduplication removes redundant near-identical frames using 16×16 thumbnails to prevent token waste on static content
• CLI overrides (--max-frames, --resolution, --fps) allow precise control when preset budgets don't match your needs

Frequently Asked Questions

How does Claude-Video calculate how many frames to extract?

Claude-Video uses an auto-FPS algorithm that divides your frame budget by the video duration, then clamps the result to a maximum of 2 FPS. This calculation happens in skills/watch/scripts/frames.py through the auto_fps and _clamp_fps functions, ensuring you never exceed your target frame count while maintaining reasonable sampling density.

What is the difference between efficient and balanced detail modes?

The efficient mode caps extraction at 50 frames using key-frame detection only, making it ideal for token-constrained workflows. The balanced mode allows up to 100 frames and uses scene-aware extraction (extract_scene_or_uniform) to select visually distinct moments. Both modes enforce the same 2 FPS maximum but differ in their selection strategy and total frame allowance.

Can I prevent Claude-Video from removing similar-looking frames?

Yes. Use the --no-dedup flag to disable perceptual deduplication. By default, Claude-Video runs dedupe_perceptual in skills/watch/scripts/frames.py to drop frames with mean-pixel differences below 2.0 (using 16×16 thumbnails), but this behavior can be disabled when analyzing content like code diffs where every frame contains unique information.

How can I reduce token costs without reducing frame count?

Lower the resolution using --resolution 256 (or any width below the default 512px). Since Claude charges image tokens based on dimensions, reducing the width proportionally reduces tokens per frame while maintaining the same number of visual reference points. Alternatively, use --detail efficient to switch to key-frame extraction, which typically selects more informative frames than uniform sampling.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}