How Auto-FPS Scaling Works in Claude-Video: Duration-Based Frame Budgeting

Auto-fps scaling in Claude-Video dynamically adjusts frame extraction rates using tiered duration thresholds in auto_fps and auto_fps_focus functions, ensuring short videos receive dense sampling while capping long videos at 100 frames and 2 FPS to control LLM token costs.

Claude-Video (bradautomates/claude-video) implements intelligent auto-fps scaling to balance visual coverage against token expenses when processing videos of varying lengths. The system uses duration-aware logic defined in skills/watch/scripts/frames.py to determine exactly how many frames to extract per second, applying conservative budgets to lengthy content while preserving high temporal resolution for brief clips.

How Auto-FPS Scaling Selects Target Frame Counts

The auto-fps scaling logic partitions videos into duration tiers, each with specific target frame counts that decrease relative to video length as clips get longer.

The Scaling Logic for Full-Video Analysis (auto_fps)

For whole-video scans, the auto_fps function (lines 22-38 in skills/watch/scripts/frames.py) applies the following duration-based rules:

• ≤30 seconds: Targets max(12, round(duration)) frames, ensuring at least 12 frames or one per second for very short content
• 31–60 seconds: Fixed target of 40 frames
• 61–180 seconds (≤3 minutes): Fixed target of 60 frames
• 181–600 seconds (≤10 minutes): Fixed target of 80 frames
• >600 seconds: Limited to max_frames (default 100)

Dense Sampling for Focused Ranges (auto_fps_focus)

When analyzing user-specified sub-ranges, auto_fps_focus (lines 41-59) applies denser sampling multipliers:

• ≤30 seconds: Targets max(10, round(duration × 6)) frames for detailed zoom-in analysis
• 31–60 seconds: Targets max(30, round(duration × 4)) frames
• 61–180 seconds: Fixed target of 60 frames
• 181–600 seconds: Fixed target of 80 frames
• >600 seconds: Limited to max_frames (default 100)

Safety Limits and FPS Clamp

After calculating the target frame count, the _clamp_fps helper (lines 49-52) enforces safety constraints defined in skills/watch/scripts/config.py:

def _clamp_fps(fps: float, duration_seconds: float, max_frames: int) -> tuple[float, int]:
    fps = min(fps, MAX_FPS)                     # never > 2 fps

    target = min(max_frames, max(1, int(round(fps * duration_seconds))))
    return fps, target

This ensures the extraction rate never exceeds MAX_FPS (2 fps) and guarantees at least one frame is emitted even for very short clips.

Implementation in the Frame Extraction Pipeline

The auto-fps scaling integrates into the extraction workflow orchestrated by skills/watch/scripts/watch.py as follows:

1. Duration Detection: The system retrieves total duration (or focused window length) via metadata
2. Strategy Selection: Calls either auto_fps for full videos or auto_fps_focus for sub-ranges
3. FPS Override: Users can manually specify --fps to bypass automatic scaling
4. FFmpeg Execution: The calculated FPS passes to extract or extract_keyframes functions that invoke ffmpeg

from pathlib import Path
from skills.watch.scripts.frames import auto_fps, auto_fps_focus, extract

video = Path("example.mp4")
duration = 45.0                      # seconds obtained from metadata

fps, target = auto_fps(duration)    # Returns ~0.89 fps, 40 frames

# For a 10-second focused window:

fps_focused, target_focused = auto_fps_focus(10.0)

# Returns 2.0 fps (capped at MAX_FPS), 20 frames

out_dir = Path("frames")
frames = extract(str(video), out_dir, fps=fps, max_frames=target)

Summary

• Auto-fps scaling uses tiered duration thresholds in skills/watch/scripts/frames.py to determine frame extraction density
• Short videos (≤30s) receive high temporal resolution (up to 6× duration for focused analysis)
• Medium videos (1–10 minutes) receive fixed frame budgets (40–80 frames)
• Long videos (>10 minutes) are capped at 100 frames maximum with 2 FPS maximum
• The _clamp_fps function enforces the 2 FPS ceiling and ensures minimum one frame output

Frequently Asked Questions

What is the maximum FPS allowed by Claude-Video's auto-fps scaling?

The system enforces a hard limit of 2 FPS through the MAX_FPS constant defined in skills/watch/scripts/config.py. Even if the duration-based calculations suggest a higher rate, the _clamp_fps function (lines 49-52 in frames.py) restricts extraction to 2 frames per second maximum.

How many frames will a 5-minute video generate with auto-fps scaling?

A 300-second (5-minute) video falls into the 181–600 second tier, which triggers a fixed target of 80 frames for both auto_fps and auto_fps_focus functions. The actual FPS will be approximately 0.27 (80 frames ÷ 300 seconds), well below the 2 FPS maximum.

Can I override the auto-fps scaling behavior?

Yes. The CLI accepts an --fps parameter that bypasses the automatic duration calculations in auto_fps and auto_fps_focus. When manually specified, the system uses your provided FPS value instead of the tiered logic, though the _clamp_fps safety limits still apply.

Why does auto-fps scaling use different targets for focused ranges versus full videos?

The auto_fps_focus function applies multipliers (6× for short clips, 4× for medium) to provide denser temporal sampling when analyzing specific sub-ranges, ensuring critical details aren't missed in zoomed analysis. In contrast, auto_fps uses conservative fixed targets for full-video scans to prevent excessive token costs when sending frames to the LLM.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}