How Auto-FPS is Calculated in Claude-Video: Adaptive Frame Budgeting by Duration and Focus Mode

Claude-Video calculates Auto-FPS using a frame budget algorithm that dynamically selects target frame counts based on video duration, applying higher density sampling when focus mode isolates specific time windows.

The bradautomates/claude-video repository implements an intelligent frame extraction system that prioritizes token efficiency over fixed sampling rates. Understanding how Auto-FPS is calculated reveals why the tool adjusts its sampling strategy based on total video length and whether a user activates focus mode for a specific segment.

The Frame Budget Philosophy

Instead of extracting frames at a constant rate, the algorithm in skills/watch/scripts/frames.py targets a specific frame budget designed to keep processing costs predictable. The system defined in auto_fps() and auto_fps_focus() calculates a target frame count first, then derives the FPS needed to distribute that budget across the actual duration.

Full-Video Scanning with auto_fps()

For analyzing complete videos, the auto_fps() function applies duration-based thresholds to determine how many frames to extract.

Duration Thresholds and Target Frames

The logic implements a tiered system that caps frame extraction for longer content:

• ≤ 30 seconds: Minimum 12 frames, up to max_frames (default 100), calculated as round(duration_seconds)
• ≤ 60 seconds: 40 frames maximum
• ≤ 180 seconds (3 minutes): 60 frames maximum
• ≤ 600 seconds (10 minutes): 80 frames maximum
• > 600 seconds: Full max_frames budget (100 frames)

if duration_seconds <= 30:
    target = min(max_frames, max(12, int(round(duration_seconds))))
elif duration_seconds <= 60:
    target = min(max_frames, 40)
elif duration_seconds <= 180:
    target = min(max_frames, 60)
elif duration_seconds <= 600:
    target = min(max_frames, 80)
else:
    target = max_frames

(see lines 27‑35 of skills/watch/scripts/frames.py)

Focus Mode Adaptation with auto_fps_focus()

When users specify a start/end window via focus mode, auto_fps_focus() assumes the isolated segment requires higher detail and applies more aggressive multipliers.

Aggressive Sampling for Focused Windows

The focused window algorithm scales the frame budget using duration multipliers:

• ≤ 5 seconds: Up to 6× duration frames (capped by max_frames)
• ≤ 15 seconds: Up to 4× duration frames
• ≤ 30 seconds: 60 frames maximum
• ≤ 60 seconds: 80 frames maximum
• ≤ 180 seconds or longer: Full max_frames budget

if duration_seconds <= 5:
    target = min(max_frames, max(10, int(round(duration_seconds * 6))))
elif duration_seconds <= 15:
    target = min(max_frames, max(30, int(round(duration_seconds * 4))))
elif duration_seconds <= 30:
    target = min(max_frames, 60)
elif duration_seconds <= 60:
    target = min(max_frames, 80)
elif duration_seconds <= 180:
    target = max_frames
else:
    target = max_frames

(see lines 46‑55 of skills/watch/scripts/frames.py)

FPS Clamping and Safety Limits

Both helpers pass their calculated rates through _clamp_fps(), which enforces a global ceiling to prevent excessive extraction.

The _clamp_fps() Safety Mechanism

The function ensures no extraction exceeds 2 FPS (MAX_FPS = 2.0) and final frame counts never exceed the configured maximum:

fps = min(fps, MAX_FPS)
target = min(max_frames, max(1, int(round(fps * duration_seconds))))

(see lines 49‑52 of skills/watch/scripts/frames.py)

This guarantees that even with aggressive multipliers in focus mode, the system never extracts more than 2 frames per second.

Integration with the Video Pipeline

The auto_fps() and auto_fps_focus() functions return a tuple (fps, target_frames) that skills/watch/scripts/watch.py uses to configure ffmpeg. The fps value determines extraction intervals, while target_frames provides the expected count for validation.

Practical Usage Example

from skills.watch.scripts.frames import auto_fps, auto_fps_focus

# Full-video scan of a 45-second clip (default max 100 frames)

fps, target = auto_fps(duration_seconds=45)
print(fps, target)   # → 0.89 fps, 40 frames (capped at 40 for 45s)

# Focused window of 8 seconds inside a longer video

fps_focused, target_focused = auto_fps_focus(duration_seconds=8)
print(fps_focused, target_focused)  # → 1.25 fps, 10 frames (6× duration, capped)

In watch.py, the orchestration logic selects which helper to call based on whether a focus range is supplied, ensuring optimal sampling density for both full analyses and targeted investigations.

Summary

• Frame budget approach: Claude-Video calculates Auto-FPS by targeting specific frame counts rather than using fixed rates, keeping token costs predictable across varying video lengths.
• Duration tiers: The auto_fps() function in skills/watch/scripts/frames.py applies five duration thresholds (30s, 60s, 180s, 600s) to cap extraction for longer videos.
• Focus mode multipliers: When analyzing focused windows, auto_fps_focus() applies aggressive multipliers (6× for clips ≤5s, 4× for ≤15s) to increase detail in short segments.
• Safety constraints: The _clamp_fps() helper enforces a hard limit of 2 FPS and ensures frame counts never exceed the configured max_frames (default 100).
• Pipeline integration: Both functions return (fps, target_frames) tuples consumed by watch.py to drive ffmpeg extraction parameters.

Frequently Asked Questions

What is the maximum FPS Claude-Video will extract?

Claude-Video enforces a hard ceiling of 2 FPS (MAX_FPS = 2.0) through the _clamp_fps() function in skills/watch/scripts/frames.py. Even when focus mode calculates higher theoretical rates for short windows, the final extraction rate never exceeds 2 frames per second.

How does focus mode change the Auto-FPS calculation?

Focus mode triggers auto_fps_focus() instead of auto_fps(), applying multipliers of 6× duration for windows ≤5 seconds and 4× duration for windows ≤15 seconds. This generates higher frame densities for critical segments while maintaining the same global caps.

Why doesn't Claude-Video use a fixed FPS for all videos?

Fixed FPS rates would generate excessive frames for long videos (increasing token costs) or insufficient frames for short clips. The frame budget algorithm ensures short videos receive adequate sampling (minimum 12 frames) while capping long videos at predictable maximums (80-100 frames).

Where is the Auto-FPS logic tested?

The adaptive frame calculation logic is validated in tests/test_frames.py, which confirms correct behavior across all duration buckets and verifies that both standard and focus modes respect the max_frames parameter and 2 FPS ceiling.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}