How Auto-FPS Calculation Works in Claude Video: Adaptive Frame Budgeting Explained

Claude Video calculates auto-fps by mapping video duration to a target frame budget rather than using a fixed rate, ensuring short clips retain high detail while long videos stay within predictable token limits for downstream processing.

The claude-video open-source project implements an intelligent auto-fps calculation system that dynamically adjusts frame extraction rates based on content length. Instead of applying a uniform frames-per-second value across all videos, the algorithm in skills/watch/scripts/frames.py allocates a specific frame budget according to duration thresholds. This adaptive approach prevents token cost explosions when processing lengthy footage while preserving granular detail in shorter segments.

Duration-Based Frame Budget Selection

The primary auto-fps logic resides in skills/watch/scripts/frames.py, where two distinct functions handle different scanning modes. Both use a tiered duration system that maps video length to a target frame count, capped by a configurable max_frames parameter (defaulting to 100).

Standard Video Scanning (auto_fps)

For full-video scans, the auto_fps() function evaluates total duration and assigns frames according to conservative thresholds (lines 27-35):

• ≤ 30 seconds: At least 12 frames, up to one frame per second of duration (e.g., a 20s video targets ~20 frames)
• ≤ 60 seconds: Hard cap of 40 frames
• ≤ 180 seconds (3 minutes): Hard cap of 60 frames
• ≤ 600 seconds (10 minutes): Hard cap of 80 frames
• > 600 seconds: Uses the full max_frames budget (100 by default)

if duration_seconds <= 30:
    target = min(max_frames, max(12, int(round(duration_seconds))))
elif duration_seconds <= 60:
    target = min(max_frames, 40)
elif duration_seconds <= 180:
    target = min(max_frames, 60)
elif duration_seconds <= 600:
    target = min(max_frames, 80)
else:
    target = max_frames

Focused Range Adaptation (auto_fps_focus)

When users specify a start/end window for detailed analysis, auto_fps_focus() applies aggressive sampling multipliers to capture finer detail in shorter segments (lines 46-55):

• ≤ 5 seconds: Up to 6× duration (e.g., 4 seconds → 24 frames maximum)
• ≤ 15 seconds: Up to 4× duration
• ≤ 30 seconds: Hard cap of 60 frames
• ≤ 60 seconds: Hard cap of 80 frames
• ≤ 180 seconds or longer: Full max_frames budget

if duration_seconds <= 5:
    target = min(max_frames, max(10, int(round(duration_seconds * 6))))
elif duration_seconds <= 15:
    target = min(max_frames, max(30, int(round(duration_seconds * 4))))
elif duration_seconds <= 30:
    target = min(max_frames, 60)
elif duration_seconds <= 60:
    target = min(max_frames, 80)
elif duration_seconds <= 180:
    target = max_frames
else:
    target = max_frames

FPS Clamping and Safety Limits

Both auto-fps helpers feed their calculated rates into _clamp_fps(), which enforces a global maximum of 2 fps (MAX_FPS = 2.0) and ensures the final frame count never exceeds max_frames (lines 49-52). This safety mechanism prevents edge cases where high multiplication factors in short focused windows might otherwise generate excessive frames.

fps = min(fps, MAX_FPS)
target = min(max_frames, max(1, int(round(fps * duration_seconds))))

The function returns a tuple (fps, target_frames), where fps represents the calculated extraction rate passed to ffmpeg for uniform frame extraction or to scene-selection pipelines.

Implementation Examples

To utilize the auto-fps calculation in your own scripts, import the functions from skills/watch/scripts/frames.py and pass the video duration in seconds:

from skills.watch.scripts.frames import auto_fps, auto_fps_focus

# Full-video scan of a 45-second clip (default max 100 frames)

fps, target = auto_fps(duration_seconds=45)
print(fps, target)   # → 0.89 fps, 40 frames (capped at 40 for 45s)

# Focused window of 8 seconds inside a longer video

fps_focused, target_focused = auto_fps_focus(duration_seconds=8)
print(fps_focused, target_focused)  # → 1.25 fps, 10 frames (6× duration, capped)

The orchestration logic in skills/watch/scripts/watch.py determines whether to invoke auto_fps() for full videos or auto_fps_focus() when a specific time range is provided, routing the resulting parameters to the appropriate extraction backend.

Summary

• Adaptive budgeting: The auto-fps system selects frame targets based on duration tiers rather than fixed rates, preventing token cost surprises with long videos while maintaining detail in short clips.
• Dual modes: auto_fps() handles full-video scans with conservative caps, while auto_fps_focus() applies aggressive multipliers for short, user-defined windows.
• Hard limits: A global MAX_FPS of 2.0 and configurable max_frames (default 100) enforced by _clamp_fps() protect against over-sampling.
• Simple integration: Functions return (fps, target_frames) tuples ready for direct use with ffmpeg or frame processing pipelines.

Frequently Asked Questions

What is the maximum frame rate allowed by Claude Video's auto-fps calculation?

The algorithm enforces a hard ceiling of 2 fps (frames per second) through the MAX_FPS constant defined in skills/watch/scripts/frames.py. Even when calculating focused ranges with high multiplication factors, the _clamp_fps() function ensures the final extraction rate never exceeds this limit, protecting downstream token budgets.

How does auto-fps calculation differ between full videos and focused time ranges?

For full videos, auto_fps() uses conservative linear caps (e.g., 40 frames max for 30-60 second videos). For focused ranges, auto_fps_focus() applies multipliers up to 6× duration for very short segments (≤5 seconds), assuming the user wants denser sampling in that specific window. Both modes respect the global max_frames limit.

Can I adjust the maximum frame budget beyond the default 100 frames?

Yes, both auto_fps() and auto_fps_focus() accept a max_frames parameter that defaults to 100. Increasing this value allows more frames for very long videos (>10 minutes), while decreasing it further constrains token costs for downstream processing.

Where does the actual frame extraction happen after auto-fps calculation?

The calculation functions return parameters to callers in skills/watch/scripts/watch.py, which orchestrates the extraction process. The calculated fps value is passed to ffmpeg for uniform frame sampling, or used to guide scene-selection algorithms for adaptive keyframe extraction.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}