How Auto-FPS Calculation Adapts Frame Rates for Videos of Any Duration

The auto-fps calculation dynamically selects frame extraction rates based on video duration tiers, capping output at 2 FPS and a configurable maximum frame budget to optimize token costs while preserving visual context.

The watch skill in the bradautomates/claude-video repository uses intelligent auto-fps calculation to determine how many frames to extract from videos of varying lengths. This system, implemented in skills/watch/scripts/frames.py, balances the need for sufficient visual detail against Claude's per-image token costs by adjusting frame density according to clip duration and user-specified ranges.

Core Logic in frames.py

The frame rate decision engine centers on two primary functions that map video duration to target frame counts. Both enforce a hard ceiling of MAX_FPS (2 fps) defined at lines 19-21 to prevent token cost explosion.

Full-Video Scanning with auto_fps

The auto_fps function (lines 22-38) handles extraction when analyzing an entire video file. It applies duration-based tiering:

• ≤ 30 seconds: Targets max(12, round(duration)) frames, ensuring short clips capture rapid action
• 30s < duration ≤ 60s: Fixed target of 40 frames
• 1 min < duration ≤ 3 min: Fixed target of 60 frames
• 3 min < duration ≤ 10 min: Fixed target of 80 frames
• > 10 minutes: Targets max_frames (default 100)

Focused Range Analysis with auto_fps_focus

When users specify a segment via --start and --end parameters, watch.py invokes auto_fps_focus (lines 41-60) instead. This function assumes the user needs higher detail within the constrained window:

• ≤ 5 seconds: Uses duration × 6 multiplier
• 5s < duration ≤ 15s: Uses duration × 4 multiplier (capped at max_frames)
• 15s < duration ≤ 30s: Fixed target of 60 frames
• 30s < duration ≤ 60s: Fixed target of 80 frames
• > 1 minute: Targets max_frames

The _clamp_fps Safety Mechanism

Both calculation paths funnel through _clamp_fps (lines 49-53), which:

1. Computes raw fps as target_frames / duration
2. Caps the result at MAX_FPS (2 fps)
3. Ensures the final frame count never exceeds the supplied max_frames parameter

Duration Tiers and Token Efficiency

The tiered approach serves specific technical constraints in vision language models. Short videos receive proportionally denser sampling—a 10-second clip may yield 12 frames (1.2 fps effective), while a 20-minute video receives only 100 frames total (0.08 fps). This dynamic density ensures transient actions in brief clips aren't missed while preventing long-form content from generating prohibitive token costs.

The focused range logic doubles down on this principle. When a user isolates a 12-second segment, the auto_fps_focus calculation attempts to extract 48 frames (4 fps), but _clamp_fps reduces this to 2 fps (24 frames) to respect the MAX_FPS safety limit.

Integration with the Watch Pipeline

The selection logic in skills/watch/scripts/watch.py (lines 55-58) determines which calculation to apply:

# From watch.py - selects appropriate fps calculator

if start_time or end_time:
    fps, target = auto_fps_focus(duration)
else:
    fps, target = auto_fps(duration)

The resulting fps value feeds directly into the ffmpeg command via -vf fps={fps}, while target validates that the extraction doesn't exceed budget constraints.

Practical Implementation Examples

from skills.watch.scripts.frames import auto_fps, auto_fps_focus

# Example 1: Full-video scan of a 45-second clip

duration = 45.0
fps, target = auto_fps(duration, max_frames=100)

# Returns: fps=0.89, target=40

print(f"Extracting {target} frames at {fps:.2f} fps")

# Example 2: Focused 12-second segment extraction

duration = 12.0
fps, target = auto_fps_focus(duration, max_frames=100)

# Calculated as 48 frames (4 fps), clamped to 2 fps = 24 frames

print(f"Extracting {target} frames at {fps:.2f} fps")

# Example 3: Long-form content (20 minutes)

duration = 20 * 60  # 1200 seconds

fps, target = auto_fps(duration, max_frames=100)

# Returns: fps=0.08, target=100 (capped at max_frames)

print(f"Extracting {target} frames at {fps:.2f} fps")

Summary

• Dual algorithms: auto_fps handles full videos while auto_fps_focus handles user-defined ranges, both residing in skills/watch/scripts/frames.py.
• Safety caps: The _clamp_fps helper enforces a maximum 2 FPS limit and respects the max_frames budget (default 100).
• Tiered targeting: Duration brackets determine frame targets, with shorter clips receiving proportionally higher frame density.
• Cost control: The 2 FPS ceiling prevents vision model token costs from scaling linearly with video length.
• Pipeline integration: watch.py selects the appropriate calculator based on the presence of --start or --end arguments.

Frequently Asked Questions

What is the maximum FPS allowed in the auto-fps calculation?

The MAX_FPS constant in skills/watch/scripts/frames.py (line 19) caps extraction at 2 FPS. Even if the duration-based math suggests a higher rate (such as in short focused ranges), the _clamp_fps function reduces the output to 2 FPS to control per-second token costs.

How does the logic differ between full-video and focused range extraction?

auto_fps assumes standard coverage needs across the entire video, using conservative frame targets for longer content. auto_fps_focus assumes the user requires granular detail within a specific window, applying multipliers like duration × 6 for segments under 5 seconds. Both functions ultimately respect the same 2 FPS and max_frames caps.

Why is there a 100-frame default limit?

The default max_frames=100 parameter prevents API rate limit issues and runaway token costs when processing long videos. At Claude's vision model pricing, 100 frames represents a predictable cost ceiling while remaining sufficient to capture key visual events across most video durations.

Where is the auto-fps calculation implemented in the codebase?

The core mathematics live in skills/watch/scripts/frames.py (functions auto_fps at lines 22-38 and auto_fps_focus at lines 41-60). The selection logic that chooses between these functions appears in skills/watch/scripts/watch.py at lines 55-58, which orchestrates the ffmpeg frame extraction command.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}