How Claude-Video's auto-fps Calculates Frame Budget Based on Video Duration

Claude-Video determines how many frames to extract by mapping video duration to tiered target counts in skills/watch/scripts/frames.py, then clamping the resulting fps to a maximum of 2.0 while enforcing a hard ceiling of 100 frames.

Claude-video is an open-source video analysis tool that intelligently manages AI token budgets by automatically calculating how many frames to sample from a video. The auto-fps calculate frame budget logic relies on two specialized helper functions that apply different density strategies for full videos versus focused time ranges, ensuring short clips retain detail while long videos stay within computational limits.

The Two-Function Architecture

The budgeting system splits video processing into two distinct modes based on whether the user specifies a time range. Both functions reside in skills/watch/scripts/frames.py and return a target frame count that is later converted into an fps value.

Full-Video Scanning with auto_fps()

The auto_fps() function handles complete video scans when no start or end timestamps are provided. As implemented in bradautomates/claude-video, this function uses duration-based tiers to set the initial frame budget:

• ≤ 30 seconds: target = min(max_frames, max(12, round(duration)))
• 31–60 seconds: target = min(max_frames, 40)
• 61–180 seconds (3 minutes): target = min(max_frames, 60)
• 181–600 seconds (10 minutes): target = min(max_frames, 80)
• > 600 seconds: target = max_frames

Focused Extraction with auto_fps_focus()

When watch.py detects user-provided start or end times, it invokes auto_fps_focus() instead. This function applies higher sampling density to preserve detail in shorter clips:

• ≤ 5 seconds: target = min(max_frames, max(10, round(duration * 6)))
• 5–15 seconds: target = min(max_frames, max(30, round(duration * 4)))
• 15–30 seconds: target = min(max_frames, 60)
• 30–60 seconds: target = min(max_frames, 80)
• 60–180 seconds: target = max_frames
• > 180 seconds: target = max_frames

FPS Clamping and Final Budget Calculation

After determining the target frame count, the private helper _clamp_fps() (lines 49-53) enforces operational limits. The function caps the extraction rate using the MAX_FPS = 2.0 constant and recalculates the final budget:

fps = min(fps, MAX_FPS)                     # line 50

target = min(max_frames, max(1, int(round(fps * duration_seconds))))   # line 51

This ensures the output never exceeds 2 fps, and the final frame count respects the user-supplied max_frames parameter (defaulting to 100).

Entry Point Logic

The high-level orchestration in skills/watch/scripts/watch.py selects the appropriate budgeting strategy:

if focused:
    fps, target = auto_fps_focus(effective_duration, max_frames=max_frames)   # line 32‑33

else:
    fps, target = auto_fps(effective_duration, max_frames=max_frames)        # line 34‑35

The boolean focused flag is set when --start or --end arguments are detected, triggering the granular sampling logic suitable for analyzing specific segments.

Practical Code Examples

Calculating Budgets Programmatically

from skills.watch.scripts.frames import auto_fps, auto_fps_focus

# 45-second full video falls in the 31-60s tier → target 40

fps, budget = auto_fps(duration_seconds=45, max_frames=100)
print(f"fps={fps:.2f}, budget={budget}")

# Output: fps=0.89, budget=40

# 7-second focused clip (5-15s tier → max(30, 28) = 30)

fps, budget = auto_fps_focus(duration_seconds=7, max_frames=100)
print(f"fps={fps:.2f}, budget={budget}")

# Output: fps=2.00, budget=14 (clamped to 2 fps × 7s)

CLI Usage

# Full video: uses auto_fps

python -m skills.watch.scripts.frames video.mp4 out/

# Focused 12-second segment: uses auto_fps_focus for higher density

python -m skills.watch.scripts.frames video.mp4 out/ --start 00:30 --end 00:42

Summary

• auto_fps() applies tiered fixed caps (12–100 frames) for full-video analysis, while auto_fps_focus() uses multipliers (up to 6× duration) for detailed segment analysis.
• The _clamp_fps() helper enforces a hard limit of 2 fps regardless of the initial budget calculation.
• Both functions respect the max_frames ceiling (default 100) to prevent token overflow.
• The logic automatically selects the appropriate function in watch.py based on whether time-range constraints are present.

Frequently Asked Questions

What is the maximum fps that claude-video's auto-fps will output?

The auto-fps system caps output at 2.0 fps via the MAX_FPS constant defined in skills/watch/scripts/frames.py. Even if the initial budget calculation suggests a higher sampling rate (for example, a 10-second clip with a target of 60 frames would suggest 6 fps), the _clamp_fps() function limits the final fps to 2.0, and the frame budget is recalculated as min(max_frames, int(round(2.0 * duration_seconds))).

How does focused mode differ from full-video mode?

Focused mode activates when users provide --start or --end timestamps, causing watch.py to invoke auto_fps_focus() instead of auto_fps(). This function allocates frames more aggressively—using multipliers like 6× duration for clips under 5 seconds and 4× for 5–15 second clips—to ensure fine-grained detail in short segments. Full-video mode uses fixed caps (40–80 frames) for medium durations, prioritizing coverage over density.

Can I change the default 100-frame maximum?

Yes. Both auto_fps() and auto_fps_focus() accept a max_frames parameter that defaults to 100 but can be overridden. When calling the functions directly, pass your desired ceiling (e.g., max_frames=50 to reduce API costs, or max_frames=200 for high-detail analysis). The CLI also respects this parameter when configured through the application settings.

Why does a 16-second focused clip get twice as many frames as a 15-second focused clip?

In auto_fps_focus(), duration tiers have discrete thresholds. A 15-second clip falls into the 5–15 second tier, which calculates max(30, round(15 * 4)) = 30 frames. A 16-second clip crosses into the 15–30 second tier, which sets a fixed target of 60 frames. This step-function approach ensures that slightly longer clips receive significantly more budget to maintain perceptual quality as the content window expands.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}