How Auto-FPS Is Calculated and Adapts to Video Duration in Claude-Video

Claude-video implements a duration-based frame budgeting algorithm that dynamically selects target frame counts from predefined thresholds, ensuring dense sampling for short videos while capping long videos to control token costs.

The claude-video repository employs an adaptive auto-fps strategy that replaces fixed frame rates with intelligent duration scaling. Implemented in skills/watch/scripts/frames.py, this algorithm calculates extraction density based on temporal buckets, keeping short segments information-rich while preventing lengthy content from generating excessive tokens for downstream processing.

Duration-Based Frame Budgeting

Unlike traditional video processing that uses constant frames-per-second values, claude-video targets specific frame budgets tailored to video length. The system selects a target number of frames (target) based on duration thresholds, then derives the effective FPS by dividing that budget by the video duration. This ensures that a 10-second clip receives proportionally more frames than a 10-minute video, maintaining visual utility without overwhelming token limits.

Full-Video Analysis with auto_fps()

For scanning entire videos, the auto_fps() function (lines 27-35 of skills/watch/scripts/frames.py) applies a five-tier duration hierarchy:

• ≤ 30 seconds: Minimum 12 frames, up to max_frames (default 100), or one frame per second if duration exceeds 12 seconds
• ≤ 60 seconds: Hard cap at 40 frames
• ≤ 180 seconds (3 min): Hard cap at 60 frames
• ≤ 600 seconds (10 min): Hard cap at 80 frames
• > 600 seconds: Uses the full max_frames budget

if duration_seconds <= 30:
    target = min(max_frames, max(12, int(round(duration_seconds))))
elif duration_seconds <= 60:
    target = min(max_frames, 40)
elif duration_seconds <= 180:
    target = min(max_frames, 60)
elif duration_seconds <= 600:
    target = min(max_frames, 80)
else:
    target = max_frames

Focused Range Analysis with auto_fps_focus()

When users specify a start/end window (handled via skills/watch/scripts/watch.py), auto_fps_focus() (lines 46-55) assumes higher detail requirements and scales the budget more aggressively for short windows:

• ≤ 5 seconds: Up to 6× duration frames (e.g., 30 frames for 5s), capped at max_frames
• ≤ 15 seconds: Up to 4× duration frames
• ≤ 30 seconds: Hard cap at 60 frames
• ≤ 60 seconds: Hard cap at 80 frames
• ≤ 180 seconds or longer: Full max_frames budget

if duration_seconds <= 5:
    target = min(max_frames, max(10, int(round(duration_seconds * 6))))
elif duration_seconds <= 15:
    target = min(max_frames, max(30, int(round(duration_seconds * 4))))
elif duration_seconds <= 30:
    target = min(max_frames, 60)
elif duration_seconds <= 60:
    target = min(max_frames, 80)
elif duration_seconds <= 180:
    target = max_frames
else:
    target = max_frames

Safety Limitations and FPS Clamping

Both functions pass their calculated rate through _clamp_fps() (lines 49-52), which enforces a global maximum of 2 FPS (MAX_FPS = 2.0) and ensures the final count never exceeds max_frames:

fps = min(fps, MAX_FPS)
target = min(max_frames, max(1, int(round(fps * duration_seconds))))

This clamping prevents pathological cases where very short durations might generate impractical frame rates, ensuring ffmpeg receives reasonable extraction parameters regardless of input length.

Implementation Example

The following example demonstrates how these functions calculate extraction parameters for different scenarios:

from skills.watch.scripts.frames import auto_fps, auto_fps_focus

# Full scan of a 45-second clip (default max_frames=100)

fps, target = auto_fps(duration_seconds=45)
print(f"Full video: {fps:.2f} FPS, {target} frames")

# Output: Full video: 0.89 FPS, 40 frames (capped at 40 for 45s duration)

# Focused 8-second window within a longer video

fps_focused, target_focused = auto_fps_focus(duration_seconds=8)
print(f"Focused range: {fps_focused:.2f} FPS, {target_focused} frames")

# Output: Focused range: 1.25 FPS, 10 frames (6x duration, then clamped)

Both functions return a tuple (fps, target_frames) that downstream orchestration passes to ffmpeg for uniform extraction or to scene-selection pipelines.

Summary

• Frame budgeting: Claude-video uses duration thresholds to select target frame counts rather than fixed FPS values, keeping short videos dense and long videos manageable.
• Dual strategies: auto_fps() handles full-video scans with conservative density, while auto_fps_focus() allocates higher frame budgets for user-specified time windows.
• Hard limits: The _clamp_fps() helper enforces a maximum 2 FPS ceiling and caps total frames at max_frames (default 100) to prevent token overflow.
• Source location: All logic resides in skills/watch/scripts/frames.py, called by skills/watch/scripts/watch.py based on whether a focus range is supplied.

Frequently Asked Questions

How does auto-fps differ from a fixed frame rate?

Auto-fps dynamically adjusts the extraction rate based on video duration, targeting specific frame budgets (e.g., 40 frames for 60-second videos) rather than extracting at constant intervals. This ensures short clips capture sufficient temporal detail while preventing long videos from generating thousands of expensive tokens.

What is the maximum number of frames claude-video will extract?

The default hard cap is 100 frames (max_frames), configurable by the caller. Additionally, no extraction exceeds 2 FPS due to the MAX_FPS constant in _clamp_fps(), ensuring even sub-second focus windows remain bounded.

Why does the focused range mode use multipliers like 6× and 4× duration?

These multipliers prioritize temporal density for short analytical windows (under 15 seconds) where users likely need granular motion analysis. For windows under 5 seconds, the 6× multiplier ensures at least 10 frames while respecting the 100-frame maximum, providing enough visual context for AI interpretation without oversampling.

Where does the orchestration logic decide which auto-fps mode to use?

The selection occurs in skills/watch/scripts/watch.py, which detects whether the user provided a start/end timestamp. If a focus range exists, it invokes auto_fps_focus(); otherwise, it defaults to auto_fps() for full-video processing, as confirmed by the unit tests in tests/test_frames.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}