Understanding the Frame Budget in Claude Video: Managing Token Costs Effectively

The frame budget in Claude Video is a duration-aware limit that controls how many frames are extracted from a video, directly capping the number of image tokens consumed—typically costing approximately 197 tokens per 512px-wide frame—to prevent excessive API costs during /watch queries.

The bradautomates/claude-video repository implements an intelligent frame sampling system to balance video comprehension with API economy. When processing video content through the /watch command, the tool extracts JPEG frames that Claude converts into image tokens, making the frame budget the primary mechanism for controlling token expenditure.

How the Frame Budget Works in Claude Video

The frame budget operates as a duration-aware allocation system defined in the repository's documentation and implementation. According to the README.md, the script calculates a target frame count based on video length, then derives an appropriate frames-per-second (FPS) value capped at 2 FPS.

Duration-Based Frame Allocation

The automatic budget scales with video length to maintain density in short clips while preventing long videos from generating prohibitive token costs:

Video Duration	Default Frame Budget	Resulting FPS
≤ 30 seconds	~12–30 frames	Up to 2 FPS
30 seconds – 1 minute	~40 frames	Up to 2 FPS
1–3 minutes	~60 frames	Up to 2 FPS
3–10 minutes	~80 frames	Up to 2 FPS
> 10 minutes	Capped by detail mode	Up to 2 FPS

This logic is implemented in skills/watch/scripts/frames.py between lines 22-38, where the auto-fps calculation selects the target frame count based on input duration.

The 2 FPS Hard Limit

Regardless of video length, the system enforces a maximum extraction rate of 2 frames per second. This cap ensures that even high-frame-rate source material cannot accidentally trigger excessive token consumption.

From Frames to Tokens: Cost Calculation

Each extracted JPEG becomes an image token input for Claude. The token calculation follows Anthropic's formula: image tokens ≈ (width × height) / 750.

With the default 512px width (typically 512 × 288 resolution, or ~147,000 pixels), each frame consumes approximately 197 tokens. Therefore, a standard 5-minute clip with an 80-frame budget generates roughly 15,700 image tokens before deduplication.

Detail Modes and Budget Overrides

The --detail flag introduces secondary caps that interact with the duration-based budget. The final frame count equals the minimum of the duration budget and the mode-specific cap.

• efficient: Maximum 50 frames (keyframe-only extraction)
• balanced: Maximum 100 frames (scene-aware sampling)
• token-burner: Uncapped (respects only the 2 FPS limit)

These constraints are documented in skills/watch/SKILL.md and enforced during the frame extraction phase.

Frame Deduplication and Final Token Count

After initial extraction, Claude Video runs a perceptual deduplication pass to remove near-identical frames. The deduplication threshold is set to a 2.0 mean-pixel difference, aggressive enough to eliminate redundant visual data while preserving meaningful scene changes. This process further reduces the final token count sent to the API.

Controlling the Frame Budget via Command Line

Users can override automatic budget selection through several command-line options in scripts/watch.py:

# Let the script calculate the duration-aware budget automatically

watch https://youtu.be/abc123

# Manually cap the maximum frames to reduce token costs

watch https://youtu.be/abc123 --max-frames 40

# Process only a specific segment for denser frame sampling

watch https://youtu.be/abc123 --start 0:45 --end 1:15

# Disable deduplication to keep every sampled frame

watch https://youtu.be/abc123 --no-dedup

# Select detail modes that impose hard caps

watch https://youtu.be/abc123 --detail efficient    # Max 50 frames

watch https://youtu.be/abc123 --detail token-burner # Uncapped budget

Summary

• The frame budget in Claude Video scales automatically with video duration, ranging from ~12 frames for short clips to 80+ frames for long content, always capped at 2 FPS.
• Each frame costs approximately 197 tokens at the default 512px width, making the budget the primary determinant of API costs.
• Detail modes (efficient, balanced, token-burner) provide hard caps of 50, 100, or unlimited frames respectively.
• Perceptual deduplication removes redundant frames after extraction, further optimizing token usage.
• Override the automatic budget using --max-frames, segment selection (--start/--end), or detail modes in scripts/watch.py.

Frequently Asked Questions

How is the frame budget calculated for videos longer than 10 minutes?

For videos exceeding 10 minutes, the duration-based budget yields to the detail mode cap. In efficient mode, extraction stops at 50 frames; in balanced mode, at 100 frames; and in token-burner mode, extraction continues up to the 2 FPS limit without an upper bound. This prevents multi-hour videos from generating millions of tokens.

Can I extract more frames than the default budget allows?

Yes. Use the --detail token-burner flag to remove the duration-based cap, or specify --max-frames with a higher number. However, remember that the 2 FPS hard limit remains active regardless of settings, as implemented in skills/watch/scripts/frames.py.

Why does the README mention 197 tokens per frame specifically?

This number derives from the resolution used by Claude Video. At the default 512px width (typically 512 × 288 pixels), the image contains approximately 147,456 pixels. Applying Anthropic's formula (width × height) / 750 yields roughly 197 tokens per frame.

Does disabling deduplication significantly increase token costs?

It can. The deduplication pass in skills/watch/scripts/frames.py removes frames with less than 2.0 mean-pixel difference, often eliminating 20-40% of extracted frames in static or slow-moving video. Using --no-dedup sends all budgeted frames to the API, potentially increasing costs proportionally.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}