How Claude Video Calculates Frame Budget Based on Video Duration

Claude Video determines the optimal frame extraction budget by mapping video duration to predefined target frame counts, then converting those targets into a frames-per-second (fps) value capped at 2.0 fps.

The system intelligently scales frame sampling density according to video length, ensuring short clips receive granular coverage while lengthy content stays within token-cost limits. This calculation is implemented in the bradautomates/claude-video repository through tiered logic that distinguishes between full-video analysis and focused range scanning.

The Two Budget Calculation Modes

Claude Video operates in two distinct modes when calculating frame budgets, each governed by a specific helper function in skills/watch/scripts/frames.py.

Full-Video Scanning with auto_fps

When analyzing an entire video without user-specified time constraints, the system invokes auto_fps (lines 27-36). This function selects target frame counts based on duration thresholds:

• ≤ 30 seconds: max(12, round(duration)) frames
• ≤ 60 seconds: 40 frames
• ≤ 180 seconds: 60 frames
• ≤ 600 seconds: 80 frames
• > 600 seconds: Uses the caller-provided max_frames parameter (default 100)

This tiered approach ensures that a 45-second video receives exactly 40 frames, while a 5-minute video gets 80 frames, preventing excessive token usage on long-form content.

Focused Range Scanning with auto_fps_focus

For user-defined time ranges (focused mode), the system employs auto_fps_focus (lines 41-57), which allocates denser sampling to shorter segments:

• ≤ 5 seconds: max(10, round(duration × 6)) frames
• ≤ 15 seconds: max(30, round(duration × 4)) frames
• ≤ 30 seconds: 60 frames
• ≤ 60 seconds: 80 frames
• > 60 seconds: Uses max_frames

This mode provides up to 6 frames per second of calculated budget for very short clips (before the 2.0 fps cap is applied), enabling detailed analysis of brief but critical segments.

FPS Capping and Final Frame Count

Both calculation modes feed into _clamp_fps, which enforces the hard constraint of MAX_FPS = 2.0 (lines 49-53). The function performs two critical operations:

1. Caps the calculated fps at 2.0 to prevent excessive frame extraction
2. Computes the final target using max(1, round(fps × duration_seconds)), ensuring the result never exceeds the caller's max_frames limit

This means a 4-second clip in focused mode might request round(4 × 6) = 24 frames theoretically, but _clamp_fps limits this to round(2.0 × 4) = 8 frames, or potentially adjusts the fps downward to meet the max_frames constraint.

Implementation Details in frames.py

The core logic resides in skills/watch/scripts/frames.py according to the source code:

• auto_fps target selection logic occupies lines 27-36
• auto_fps_focus target selection logic spans lines 41-57
• FPS clamping and final target computation are handled in lines 49-53

The watch.py script serves as the CLI entry point that orchestrates these calculations based on user arguments, while tests/test_frames.py verifies the budget behavior across various durations and modes.

Practical Code Examples

Calculating Budget for Full-Video Analysis

from skills.watch.scripts.frames import auto_fps, get_metadata

meta = get_metadata("example.mp4")
duration = meta["duration_seconds"]  # e.g., 45.2 seconds

fps, target = auto_fps(duration, max_frames=100)

print(f"fps={fps:.2f}, target_frames={target}")

# Output: fps=0.88, target_frames=40 (applying the ≤60s tier)

Calculating Budget for Focused Range

from skills.watch.scripts.frames import auto_fps_focus

duration = 4.3  # seconds in user-selected window

fps, target = auto_fps_focus(duration, max_frames=100)

print(f"fps={fps:.2f}, target_frames={target}")

# Output: fps=2.00, target_frames=10 (denser sampling for ≤5s clips)

Integrating Budget into Extraction Pipeline

from pathlib import Path
from skills.watch.scripts.frames import extract, get_metadata, auto_fps

video_path = "lecture.mov"
out_dir = Path("frames")

meta = get_metadata(video_path)
fps, target = auto_fps(meta["duration_seconds"], max_frames=100)

frames = extract(
    video_path,
    out_dir,
    fps=fps,
    resolution=512,
    max_frames=target,
)

print(f"Extracted {len(frames)} frames")

Summary

• Claude Video uses duration-based tiers to determine target frame counts before calculating fps.
• auto_fps handles full-video scans with breakpoints at 30s, 60s, 180s, and 600s.
• auto_fps_focus provides denser sampling for user-defined ranges with breakpoints at 5s, 15s, 30s, and 60s.
• _clamp_fps enforces a hard limit of 2.0 fps and ensures the final frame count respects max_frames.
• The implementation in skills/watch/scripts/frames.py (lines 27-57) ensures short videos receive proportional coverage while limiting token costs for long content.

Frequently Asked Questions

What is the maximum frame extraction rate in Claude Video?

Claude Video enforces a hard upper bound of 2.0 fps through the _clamp_fps function in skills/watch/scripts/frames.py. Even if the budget calculation suggests a higher rate (such as 6 fps for very short focused segments), the system caps extraction at 2 frames per second to manage downstream processing costs.

How does Claude Video handle videos longer than 10 minutes?

For videos exceeding 600 seconds (10 minutes), both auto_fps and auto_fps_focus defer to the caller-provided max_frames parameter, which defaults to 100 frames. This prevents the frame budget from growing linearly with video length and keeps token usage predictable for long-form content.

Why are there different calculation modes for full videos versus focused ranges?

The dual-mode design allows Claude Video to optimize for different use cases. Full-video scanning (auto_fps) allocates modest frame budgets across long durations for general understanding, while focused range scanning (auto_fps_focus) provides denser temporal resolution for brief, user-selected segments where granular detail is required. Both modes ultimately respect the 2.0 fps cap defined in _clamp_fps.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}