How the Auto-FPS Frame Budget Algorithm Works for Videos of Different Lengths

The auto-fps frame budget algorithm dynamically selects a frame rate that respects a hard ceiling of 2 FPS and a user-defined maximum frame count, applying duration-based thresholds that provide denser sampling for short clips while capping long videos to prevent budget overruns.

The bradautomates/claude-video repository implements this adaptive sampling strategy in its frame extraction pipeline. The auto-fps frame budget algorithm intelligently balances capture density against computational limits by scaling the target frame count according to video duration. This ensures that brief focus windows maintain high temporal resolution while lengthy footage never exceeds the configured max_frames limit.

Core Logic of the Frame Budget

The algorithm operates through a three-step process defined in skills/watch/scripts/frames.py. First, it determines a target frame count based on the video's duration_seconds. Second, it converts this target into an effective FPS value. Finally, it applies _clamp_fps to enforce hard constraints.

The MAX_FPS Ceiling

Regardless of video length, the system never extracts frames faster than 2 FPS (frames per second). This global limit is defined as MAX_FPS = 2.0 in the configuration. The _clamp_fps function ensures this ceiling is respected:

def _clamp_fps(fps: float, duration_seconds: float, max_frames: int) -> tuple[float, int]:
    fps = min(fps, MAX_FPS)                     # never exceed 2 fps

    target = min(max_frames, max(1, int(round(fps * duration_seconds))))
    return fps, target

This guarantees that even for very short clips, the extraction rate never exceeds the hardware and API limits defined by the repository's configuration.

Duration-Based Thresholds

The auto_fps function implements tiered logic that maps video length to specific frame budgets. According to the source code in skills/watch/scripts/frames.py (lines 122-138), the thresholds are:

• ≤ 30 seconds: max(12, round(duration_seconds)) frames, capped at max_frames
• 31–60 seconds: Fixed budget of 40 frames
• 61–180 seconds (3 minutes): Fixed budget of 60 frames
• 181–600 seconds (10 minutes): Fixed budget of 80 frames
• > 600 seconds: Full max_frames budget

The resulting FPS is calculated as target / duration_seconds, then clamped to the 2 FPS maximum. This creates an inverse relationship where shorter videos receive proportionally denser sampling.

Focused Range Sampling with auto_fps_focus

When users specify a start/end timestamp, the system switches to auto_fps_focus (defined in skills/watch/scripts/frames.py lines 141-158). This variant uses steeper scaling for brief analysis windows to ensure critical moments receive adequate representation:

• ≤ 5 seconds: max(10, round(duration_seconds × 6))
• 5–15 seconds: max(30, round(duration_seconds × 4))
• 15–30 seconds: Fixed budget of 60 frames
• 30–60 seconds: Fixed budget of 80 frames
• > 60 seconds: Full max_frames budget

This aggressive scaling ensures that 3-second clips capture up to 18 frames (6 FPS before clamping to 2 FPS), while the standard auto_fps would allocate only 3 frames for the same duration if treated as a full video.

Integration in the Watch Pipeline

The CLI entry point in skills/watch/scripts/watch.py (lines 55-58) selects the appropriate algorithm based on user input:

if focused:
    fps, target = auto_fps_focus(effective_duration, max_frames=budget_cap)
else:
    fps, target = auto_fps(effective_duration, max_frames=budget_cap)

The budget_cap value originates from skills/watch/scripts/config.py, which defines the default frame_cap settings. This integration ensures that the frame budget respects both algorithmic thresholds and user-configured limits.

Practical Code Examples

You can invoke these functions directly to preview the FPS and frame allocations for specific durations:

from skills.watch.scripts.frames import auto_fps, auto_fps_focus

# Full-video sampling (no focus window)

print(auto_fps(12, max_frames=100))    # → (2.0, 24)  # 12s video: 2 FPS, 24 frames

print(auto_fps(300, max_frames=100))  # → (0.33, 100) # 5min video: 0.33 FPS, 100 frames

# Focused window sampling (e.g., 0-10 second range)

print(auto_fps_focus(10, max_frames=100))  # → (2.0, 20)  # 10s focus: 2 FPS, 20 frames

These examples demonstrate how a 12-second clip receives 24 frames (dense sampling), while a 300-second (5-minute) video receives only 100 frames (0.33 FPS) to maintain the budget cap.

Summary

• The auto-fps frame budget algorithm applies duration-based thresholds to determine optimal frame counts, ensuring short videos receive dense sampling while long videos respect budget limits.
• Hard ceiling of 2 FPS (MAX_FPS) is enforced by _clamp_fps regardless of computed values.
• Dual-mode operation: auto_fps handles full-video processing, while auto_fps_focus provides aggressive scaling for short timestamp ranges.
• Implementation resides in skills/watch/scripts/frames.py, with configuration managed in skills/watch/scripts/config.py and invocation logic in skills/watch/scripts/watch.py.
• Guaranteed minimum: Both functions ensure at least one frame is extracted via the max(1, ...) clause in _clamp_fps.

Frequently Asked Questions

What is the maximum frame rate the auto-fps algorithm will ever return?

The algorithm will never return a frame rate higher than 2.0 FPS (frames per second). This MAX_FPS constant acts as a hard ceiling applied in the _clamp_fps function, ensuring that even calculations for very short videos or focused windows are capped at 2 FPS before extraction begins.

How does the algorithm prevent exceeding the user-defined frame budget?

The _clamp_fps function takes the computed target frame count and applies min(max_frames, ...) to ensure the final value never exceeds the user-supplied max_frames parameter. Additionally, tiered thresholds in auto_fps and auto_fps_focus provide conservative defaults that scale sub-linearly with video duration, naturally preventing budget overruns on long content.

When does the system use auto_fps_focus instead of auto_fps?

The system selects auto_fps_focus when the user provides a specific time range (start and end timestamps) for analysis, as implemented in skills/watch/scripts/watch.py. This focused variant applies steeper multipliers (up to 6× duration) for very short windows to ensure adequate temporal resolution in brief clips, whereas auto_fps is used for full-video processing with more conservative scaling.

Why does a 10-second video get more frames per second than a 10-minute video?

The algorithm implements duration-adaptive sampling to balance detail against computational cost. Short clips (≤ 30 seconds) trigger the max(12, round(duration)) logic, which yields higher effective FPS values, while videos over 600 seconds default to the full max_frames budget divided by duration, resulting in lower FPS. This ensures that brief moments receive dense coverage without allowing lengthy videos to consume excessive API tokens or processing time.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}