How the Auto-FPS Algorithm Calculates Frame Targets for Full-Video vs Focused Mode in claude-video

The auto-FPS algorithm uses distinct piece-wise schedules in skills/watch/scripts/frames.py to determine frame targets, capping at 2 FPS via _clamp_fps, with full-video mode optimized for coverage and focused mode prioritizing density for detailed analysis.

The bradautomates/claude-video repository implements an intelligent frame sampling system that balances token costs against visual fidelity. By automatically adjusting frames per second based on video duration and user intent, the algorithm ensures you never exceed your budget while maximizing detail where it matters. This guide explains exactly how the auto_fps and auto_fps_focus functions derive their targets and when each mode activates.

Core Implementation in frames.py

The entire auto-FPS logic resides in skills/watch/scripts/frames.py, which exposes two public helpers consumed by the CLI driver in watch.py. Both helpers return a tuple of (fps, target_frames) and rely on a shared clamping utility to enforce hard limits.

FPS Capping with _clamp_fps

Before any frame count is finalized, the _clamp_fps utility enforces the global ceiling. As implemented at lines 49-53 of frames.py, this function ensures the calculated FPS never exceeds MAX_FPS = 2.0 and that the final frame count respects the user-supplied budget:

def _clamp_fps(fps: float, duration_seconds: float, max_frames: int) -> tuple[float, int]:
    fps = min(fps, MAX_FPS)                               # ← caps at 2 fps

    target = min(max_frames, max(1, int(round(fps * duration_seconds))))
    return fps, target

This safeguard guarantees that even aggressive schedules cannot breach the 2 FPS ceiling, keeping generation costs bounded while preventing empty results.

Full-Video Mode: auto_fps

When the CLI scans an entire video without explicit --start or --end timestamps, it invokes auto_fps (lines 22-38). This function implements a sparse sampling strategy optimized for long-form content overview, using a piece-wise schedule that maps duration buckets to desired frame counts:

Duration (seconds)	Desired frames (target)
≤ 30	max(12, round(duration))
≤ 60	40
≤ 180 (3 min)	60
≤ 600 (10 min)	80
> 600	max_frames (the budget)

The implementation derives the target, then divides by duration to yield a raw FPS value before clamping:

def auto_fps(duration_seconds: float, max_frames: int = 100) -> tuple[float, int]:
    if duration_seconds <= 0:
        return 1.0, 1
    if duration_seconds <= 30:
        target = min(max_frames, max(12, int(round(duration_seconds))))
    elif duration_seconds <= 60:
        target = min(max_frames, 40)
    elif duration_seconds <= 180:
        target = min(max_frames, 60)
    elif duration_seconds <= 600:
        target = min(max_frames, 80)
    else:
        target = max_frames
    return _clamp_fps(target / duration_seconds, duration_seconds, max_frames)

Focused Mode: auto_fps_focus

When the user provides a time range, the CLI switches to auto_fps_focus (lines 41-58), which allocates a "denser" frame budget per second because the intent is zooming in for detail. The schedule accelerates sampling for short clips:

Duration (seconds)	Desired frames (target)
≤ 5	max(10, round(duration × 6))
≤ 15	max(30, round(duration × 4))
≤ 30	60
≤ 60	80
≤ 180	max_frames
> 180	max_frames

This aggressive allocation ensures that a 10-second focused clip receives up to 40 frames, whereas the same duration in full-video mode would only receive 10-12 frames:

def auto_fps_focus(duration_seconds: float, max_frames: int = 100) -> tuple[float, int]:
    if duration_seconds <= 0:
        return min(MAX_FPS, 2.0), 2
    if duration_seconds <= 5:
        target = min(max_frames, max(10, int(round(duration_seconds * 6))))
    elif duration_seconds <= 15:
        target = min(max_frames, max(30, int(round(duration_seconds * 4))))
    elif duration_seconds <= 30:
        target = min(max_frames, 60)
    elif duration_seconds <= 60:
        target = min(max_frames, 80)
    elif duration_seconds <= 180:
        target = max_frames
    else:
        target = max_frames
    return _clamp_fps(target / duration_seconds, duration_seconds, max_frames)

CLI Integration in watch.py

The mode selection logic lives in skills/watch/scripts/watch.py (lines 30-35), where the script inspects whether boundary arguments were provided:

focused = start_sec is not None or end_sec is not None
if focused:
    fps, target = auto_fps_focus(effective_duration, max_frames=max_frames)
else:
    fps, target = auto_fps(effective_duration, max_frames=max_frames)

If the user manually supplies --fps, that value overrides the calculated rate, but the target frame count remains subject to the budget constraints defined in _clamp_fps.

Practical Code Examples

You can import and test the algorithm directly to preview how it budgets frames for different scenarios:

from skills.watch.scripts.frames import auto_fps, auto_fps_focus

# Full-video: 2-minute clip, budget 100 frames

fps, target = auto_fps(duration_seconds=120, max_frames=100)
print(f"Full-video → fps={fps:.2f}, target={target}")

# Focused: user requested 0-10 s of the same clip

fps_f, target_f = auto_fps_focus(duration_seconds=10, max_frames=100)
print(f"Focused → fps={fps_f:.2f}, target={target_f}")

Typical output demonstrates the density difference:

Full-video → fps=0.83, target=100
Focused → fps=2.00, target=20

Notice that for the 10-second focused window, the algorithm hits the 2 FPS ceiling, allocating 20 frames rather than the sparse sampling used for broad scans.

Summary

• Dual-mode architecture: auto_fps handles full-video scans with sparse sampling, while auto_fps_focus provides dense sampling for time-boxed analysis.
• Piece-wise scheduling: Full-video buckets are 30s, 60s, 180s, and 600s; focused mode uses tighter 5s, 15s, 30s, and 60s thresholds with multipliers up to 6×.
• Hard FPS ceiling: _clamp_fps enforces MAX_FPS = 2.0 regardless of mode, ensuring the final frame count never exceeds the user budget.
• CLI automation: watch.py automatically selects the appropriate helper based on the presence of --start or --end arguments.

Frequently Asked Questions

What is the maximum FPS the auto-FPS algorithm will ever return?

According to the source code in skills/watch/scripts/frames.py, the _clamp_fps function explicitly caps the calculated rate at 2.0 FPS via min(fps, MAX_FPS). This ceiling applies to both full-video and focused modes.

How does the algorithm decide whether to use full-video or focused mode?

The CLI driver in watch.py checks if start_sec or end_sec is not None. If either timestamp is provided, it calls auto_fps_focus; otherwise, it defaults to auto_fps. This determination happens before any frame extraction begins.

Why does a 10-second focused clip get more frames than a 10-second segment of a full-video scan?

The auto_fps_focus schedule applies multipliers (up to 4× for 10 seconds) specifically because the user has signaled intent to examine a specific window in detail. In contrast, auto_fps treats short clips under 30 seconds as part of a broader overview, allocating only up to max(12, duration) frames to conserve budget for longer content.

Can I override the auto-FPS calculation while keeping the frame budget?

Yes. When calling the extraction pipeline, you can pass a manual --fps value, which overrides the calculated rate. However, the final frame target is still processed through _clamp_fps, ensuring the result respects your max_frames budget and never exceeds 2 FPS.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}