How the Frame Budget System Scales for Different Video Lengths in Claude-Video

The frame budget system in Claude-Video uses piece-wise duration tables to maintain constant token costs per minute by capping total extracted frames at 100 for long videos while providing higher frame density for short clips.

The bradautomates/claude-video repository implements an intelligent frame extraction strategy in its Watch skill to prevent LLM query costs from exploding as video length increases. This system calculates a dynamic frame budget based on video duration, ensuring that whether you process a 30-second clip or a 2-hour movie, the token consumption remains predictable and cost-effective.

Core Frame Budget Components

auto_fps() for Full-Video Analysis

Located in skills/watch/scripts/frames.py, the auto_fps() function determines the extraction parameters for complete video scans. It returns a target frame count and safe FPS value capped at MAX_FPS = 2.0.

auto_fps_focus() for Time-Range Analysis

When users specify start/end timestamps via command-line arguments, auto_fps_focus() (also in skills/watch/scripts/frames.py) activates to provide a denser frame budget. This focused mode allocates more frames per second for short segments since the analysis concentrates on a specific temporal window.

Piece-Wise Scaling Rules

The scaling logic employs duration-based tables that map video length to target frame counts. Both functions respect a configurable max_frames parameter (default 100) that serves as an absolute ceiling.

Full-Video Budget Table

For complete video processing, auto_fps() applies these thresholds:

• ≤ 30 seconds: min(max_frames, max(12, round(duration))) — Guarantees at least 12 frames for very short content.
• ≤ 60 seconds: min(max_frames, 40) — Fixed modest allocation for sub-minute clips.
• ≤ 180 seconds: min(max_frames, 60) — Moderate increase for three-minute videos.
• ≤ 600 seconds: min(max_frames, 80) — Extended budget up to ten minutes.

• 600 seconds: max_frames (100) — Hard cap for long-form content.

Focused-Mode Budget Table

For segmented analysis, auto_fps_focus() uses more aggressive density:

• ≤ 5 seconds: min(max_frames, max(10, round(duration × 6))) — Up to 6 fps for micro-clips.
• ≤ 15 seconds: min(max_frames, max(30, round(duration × 4))) — Up to 4 fps for short segments.
• ≤ 30 seconds: min(max_frames, 60) — Dense sampling for half-minute windows.
• ≤ 60 seconds: min(max_frames, 80) — High detail for minute-long focuses.
• ≤ 180 seconds: max_frames — Ceiling reached at three minutes.

• 180 seconds: max_frames — Maintains cap for longer ranges.

FPS Clamping and Final Calculation

After selecting the raw target from the duration table, both functions invoke _clamp_fps() to enforce physical constraints:

1. Clamp FPS to MAX_FPS = 2.0
2. Calculate final target: min(max_frames, max(1, round(fps × duration)))

This ensures the system scales linearly for short videos (higher temporal resolution) and log-linearly for long videos (frame count plateaus at the maximum budget). The resulting parameters feed directly into the ffmpeg extraction pipeline via extract() or extract_scene_or_uniform().

Practical Implementation Examples

from pathlib import Path
from skills.watch.scripts.frames import auto_fps, auto_fps_focus, extract

# Example 1 – Full‑video budget for a 45‑second clip (default max_frames=100)

duration = 45.0
fps, target = auto_fps(duration)
print(f"Full video → fps={fps:.2f}, target frames={target}")

# → fps≈0.88, target frames=40 (fits the “≤ 60 s → 40” rule)

# Example 2 – Focused‑mode budget for a 12‑second snippet

focus_duration = 12.0
fps_f, target_f = auto_fps_focus(focus_duration)
print(f"Focused → fps={fps_f:.2f}, target frames={target_f}")

# → fps≈1.33, target frames=40 (denser than the full‑video case)

# Example 3 – Extract frames using the calculated budget

video_path = "/path/to/video.mp4"
out_dir = Path("./frames")
extract(video_path, out_dir, fps=fps, max_frames=target)

The command-line entry point skills/watch/scripts/watch.py automatically selects between these modes based on the presence of --start and --end arguments.

Summary

• The frame budget system uses piece-wise duration tables in skills/watch/scripts/frames.py to map video length to target frame counts.
• Full-video mode (auto_fps()) provides conservative density suitable for long content analysis.
• Focused mode (auto_fps_focus()) delivers higher frame rates for short time ranges, capping at 180 seconds.
• A hard ceiling of 100 frames (configurable via max_frames) prevents cost explosion on long videos.
• The _clamp_fps() function enforces a maximum extraction rate of 2.0 FPS regardless of duration.

Frequently Asked Questions

What is the maximum number of frames the system will extract from a video?

The default maximum is 100 frames, controlled by the max_frames parameter in auto_fps() and auto_fps_focus(). For videos longer than 10 minutes, the budget hits this ceiling regardless of duration, ensuring predictable token costs.

How does the system decide between full-video and focused mode?

The CLI entry point watch.py checks for --start and --end arguments. If present, it calls auto_fps_focus() to allocate a denser frame budget appropriate for the specified time range. Without these arguments, auto_fps() handles the calculation for the entire video duration.

Why is the FPS capped at 2.0 in the frame budget calculation?

The MAX_FPS = 2.0 constant in skills/watch/scripts/frames.py prevents excessive frame extraction that would exceed reasonable token budgets. After calculating the theoretical FPS needed to achieve the target frame count, _clamp_fps() enforces this ceiling to maintain cost efficiency.

Does the frame budget scale linearly with video duration?

No, it scales piece-wise linear for short durations then plateaus at the max_frames cap. Short videos (under 30 seconds) receive proportional frame allocations, while videos over 10 minutes are fixed at 100 frames total, creating a log-linear cost curve that keeps LLM query expenses manageable.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}