How the Auto-FPS Frame Budget System Adjusts Frame Counts Based on Video Duration

The auto-FPS system in claude-video scales frame rates inversely with video length: short clips (≤30s) get 12–30 frames at ~1–2 FPS, while long videos (>10 min) are capped at a configurable max_frames budget (default 100), ensuring LLM token limits are never exceeded.

The auto-FPS frame budget system in the bradautomates/claude-video repository dynamically adjusts how many frames are extracted from a video based on its duration. This prevents downstream LLM context windows from being overwhelmed while preserving visual density for shorter, detail-critical segments. The implementation lives primarily in skills/watch/scripts/frames.py.

Core Frame Budget Functions

Two helper functions drive the duration-based adjustment logic. Both return a tuple of (fps, target_frames) and internally delegate to _clamp_fps, which enforces a hard ceiling of MAX_FPS = 2.0 and ensures the final count never exceeds max_frames.

auto_fps — Full-Video Frame Budget

Located at lines 22–38 of frames.py, auto_fps handles scans across an entire video when no start/end range is specified. It uses tiered duration thresholds with fixed frame targets:

def auto_fps(duration_seconds: float, max_frames: int = 100) -> tuple[float, int]:
    if duration_seconds <= 0:
        return 1.0, 1
    if duration_seconds <= 30:
        target = min(max_frames, max(12, int(round(duration_seconds))))   # 12–30 frames

    elif duration_seconds <= 60:
        target = min(max_frames, 40)                                      # 40 frames

    elif duration_seconds <= 180:   # ≤ 3 min

        target = min(max_frames, 60)                                      # 60 frames

    elif duration_seconds <= 600:   # ≤ 10 min

        target = min(max_frames, 80)                                      # 80 frames

    else:
        target = max_frames                                                # cap only

    return _clamp_fps(target / duration_seconds, duration_seconds, max_frames)

The duration-to-frame mapping works as follows:

• ≤ 30 seconds: Target 12–30 frames (roughly 1 frame per second, minimum 12)
• 31–60 seconds: Fixed 40-frame budget (~0.7 FPS)
• 1–3 minutes: Fixed 60-frame budget (~0.33–1 FPS)
• 3–10 minutes: Fixed 80-frame budget (~0.13–0.44 FPS)
• > 10 minutes: Strict max_frames ceiling only (default 100)

The final FPS is computed as target / duration_seconds, then clamped to respect MAX_FPS.

auto_fps_focus — Focused-Range Frame Budget

For user-specified time ranges (via --start/--end), auto_fps_focus (lines 41–59) allocates a denser frame budget because the user is explicitly "zooming in" for detail:

def auto_fps_focus(duration_seconds: float, max_frames: int = 100) -> tuple[float, int]:
    if duration_seconds <= 0:
        return min(MAX_FPS, 2.0), 2
    if duration_seconds <= 5:
        target = min(max_frames, max(10, int(round(duration_seconds * 6))))   # up to 6 FPS

    elif duration_seconds <= 15:
        target = min(max_frames, max(30, int(round(duration_seconds * 4))))   # up to 4 FPS

    elif duration_seconds <= 30:
        target = min(max_frames, 60)
    elif duration_seconds <= 60:
        target = min(max_frames, 80)
    elif duration_seconds <= 180:
        target = max_frames
    else:
        target = max_frames
    return _clamp_fps(target / duration_seconds, duration_seconds, max_frames)

Key differences from auto_fps:

• ≤ 5 seconds: Up to 6 FPS (30 frames for 5s), capped by max_frames
• 5–15 seconds: Up to 4 FPS denser sampling
• 15–60 seconds: Stepped 60→80 frame budgets
• > 60 seconds: Reverts to max_frames cap

Integration in the Watch Pipeline

The entry point at skills/watch/scripts/watch.py (lines 55–59) selects the appropriate budget function based on whether a focused range was requested:

if focused:
    fps, target = auto_fps_focus(effective_duration, max_frames=budget_cap)   # zoomed detail

else:
    fps, target = auto_fps(effective_duration, max_frames=budget_cap)       # full coverage

The calculated fps and target values are then forwarded to extraction engines (extract, extract_keyframes, extract_scene_or_uniform). The frame budget is always respected as an absolute ceiling, while the visual density scales appropriately for the content length.

Frame Budget Comparison Table

Video Duration	Full-Video (auto_fps)	Focused Range (auto_fps_focus)
0–5 s	5–12 frames	Up to 30 frames (6 FPS)
5–15 s	12–15 frames	30–60 frames (up to 4 FPS)
15–30 s	15–30 frames	60 frames
30–60 s	40 frames	80 frames
1–3 min	60 frames	60 frames → max_frames
3–10 min	80 frames	max_frames
> 10 min	max_frames (default 100)	max_frames

Practical Usage Examples

from skills.watch.scripts.frames import auto_fps, auto_fps_focus

# Full 45-second video → sparse sampling for coverage

fps, target = auto_fps(duration_seconds=45)

# Returns: (0.89, 40)

# Focused 5-second clip → dense sampling for detail

fps, target = auto_fps_focus(duration_seconds=5)

# Returns: (2.0, 10)  # capped at MAX_FPS

# Long 15-minute video → strict budget enforcement

fps, target = auto_fps(duration_seconds=900, max_frames=120)

# Returns: (~0.13, 120)  # fps clamped, frames capped

Key Implementation Files

• skills/watch/scripts/frames.py: Contains auto_fps, auto_fps_focus, _clamp_fps, and the extraction pipeline
• skills/watch/scripts/watch.py: Entry point that routes to the appropriate budget function
• skills/watch/scripts/config.py: Defines default max_frames values per detail level (indirectly)

Summary

• The auto-FPS frame budget system uses duration-tiered logic to balance visual coverage against LLM token constraints
• auto_fps provides economical sampling for full-video context; auto_fps_focus allocates denser frames for user-selected ranges
• Both functions enforce hard limits via _clamp_fps: maximum 2.0 FPS and never exceeding max_frames
• Default budget accommodates ~100 frames; configurable per invocation
• Frame density scales inversely with duration: short clips maintain high fidelity, long videos stay within safe consumption limits

Frequently Asked Questions

What is the maximum frame rate the auto-FPS system will ever return?

The system caps FPS at MAX_FPS = 2.0 regardless of video duration or focus mode. This ceiling is enforced by _clamp_fps in frames.py to prevent excessive frame generation that would waste tokens without improving LLM comprehension.

How do I increase the frame budget for a very long video?

Pass a higher max_frames value to either auto_fps or auto_fps_focus. The default is 100, but this is configurable per call. For videos exceeding 10 minutes, this is the only lever available—the tiered logic yields entirely to your specified budget cap.

Why does the focused mode give more frames for short clips?

auto_fps_focus assumes the user has explicitly selected a time range because that segment contains important detail. The system responds with denser sampling (up to 6 FPS for ≤5s) to ensure no critical visual information is missed, whereas auto_fps optimizes for broad coverage across the full timeline.

Where is the frame budget actually enforced during extraction?

The fps and target values computed by auto_fps or auto_fps_focus are passed to extraction functions in frames.py (extract, extract_keyframes, extract_scene_or_uniform). These functions use the target count to terminate or throttle frame selection, ensuring the budget is never exceeded regardless of the underlying video complexity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}