How the Frame-Budget System Scales with Video Duration in the Claude Video Watch Skill

The watch skill uses tiered duration thresholds and configurable caps to adaptively limit frame extraction, scaling from 12 frames for 30-second clips to 100 frames for videos over 10 minutes.

The frame-budget system in bradautomates/claude-video dynamically balances visual coverage against token cost. Located in skills/watch/scripts/frames.py, this system uses two specialized functions—auto_fps for full-video analysis and auto_fps_focus for user-specified ranges—to compute extraction parameters based on video length.

How the Frame-Budget Scaling Works

The budget system divides videos into five duration tiers, applying progressively higher fixed caps until reaching the user-defined max_frames limit (default 100).

Full-Video Analysis: auto_fps

For complete video scans, auto_fps (lines 22-36) applies these rules:

Duration	Frame Target Calculation	Result
≤ 30 s	min(max_frames, max(12, round(duration)))	12–30 frames
≤ 60 s	min(max_frames, 40)	40 frames
≤ 180 s	min(max_frames, 60)	60 frames
≤ 600 s	min(max_frames, 80)	80 frames
> 600 s	max_frames	100 frames (default cap)

Short clips receive proportional coverage (roughly 1 frame per second), while longer videos hit fixed ceilings to prevent runaway token costs.

Focused-Range Analysis: auto_fps_focus

When users specify start/end timestamps, auto_fps_focus (lines 41-57) allocates a denser budget for the shorter effective duration:

Duration	Frame Target Calculation	Result
≤ 30 s	min(max_frames, max(10, round(duration × 6)))	Up to 180 frames before cap
≤ 60 s	min(max_frames, max(30, round(duration × 4)))	Up to 240 frames before cap
≤ 180 s	min(max_frames, 60)	60 frames
≤ 600 s	min(max_frames, 80)	80 frames
> 600 s	max_frames	100 frames

The multiplier-based formulas (×6 and ×4) ensure zoomed-in segments receive richer visual detail than equivalent-length full videos.

FPS Clamping and Final Calculation

Both helpers pass through _clamp_fps (lines 49-53), which:

1. Computes raw FPS as target_frames / duration
2. Enforces hard ceiling: MAX_FPS = 2.0
3. Recalculates final frame count from clamped FPS

This prevents excessive extraction rates that would bloat token usage without adding analytical value.

Integration with the Watch Skill

The main entry point in skills/watch/scripts/watch.py (lines 55-59) routes to the appropriate helper based on the focused flag:

from skills.watch.scripts.frames import auto_fps, auto_fps_focus

# Full video: 45-second clip → ~40 frames

fps, target = auto_fps(duration_seconds=45)
print(fps, target)  # 0.89, 40

# 10-second focused range → denser sampling

fps_f, target_f = auto_fps_focus(duration_seconds=10, max_frames=100)
print(fps_f, target_f)  # 2.0 (capped), 20 frames after clamp recalculation

Note: Despite auto_fps_focus targeting 60 frames for a 10-second window, _clamp_fps caps FPS at 2.0, yielding 20 final frames.

Command-Line Usage

# Default full-video budget

watch https://example.com/video.mp4

# Focused range triggers denser allocation

watch https://example.com/video.mp4 --start 00:01:00 --end 00:01:10

CLI output confirms the budget selection:

[watch] extracting scene-aware frames over 00:01:00-00:01:10 (10.0s) (target 60, cap 100)…

Key Design Characteristics

• Linear scaling for short durations (≤30s) preserves detail where it matters most
• Tiered caps create predictable cost curves for medium-length content
• Hard maximum (max_frames) gives users explicit control over worst-case token spend
• Focus mode boost prioritizes visual fidelity in user-selected segments

Summary

• The frame-budget system uses duration-based tiers with fixed caps to scale from 12 to 100 frames
• auto_fps in skills/watch/scripts/frames.py handles full-video scans with conservative sampling
• auto_fps_focus applies denser multipliers (×6, ×4) for user-specified ranges
• _clamp_fps enforces MAX_FPS = 2.0 regardless of target calculations
• The watch.py entry point selects helpers via the focused flag and passes results to the extraction engine

Frequently Asked Questions

How does the frame budget change for videos longer than 10 minutes?

Videos exceeding 600 seconds always receive exactly max_frames (default 100), regardless of additional length. This creates a fixed cost ceiling that prevents token usage from scaling with video duration.

Why does focused mode use multipliers instead of fixed targets?

The ×6 and ×4 multipliers in auto_fps_focus ensure short user-selected windows receive proportionally richer sampling. A 5-second focus range targets 30 frames (5 × 6), while the same duration in full-video mode would receive only 12 frames.

Can users override the default 100-frame maximum?

Yes. Both auto_fps and auto_fps_focus accept a max_frames parameter. Lower values reduce token costs; higher values increase visual coverage at the risk of exceeding context-window limits.

What happens if the calculated FPS exceeds 2.0?

The _clamp_fps function forces FPS down to 2.0 and recalculates the final frame count. This protects against edge cases where duration formulas produce unsustainably high extraction rates.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}