How Frame Budget Scales with Video Duration in Claude-Video

The frame budget in Claude-Video scales using a tiered duration-based algorithm that allocates 12-40 frames for videos under one minute, 60 frames for 1-3 minutes, 80 frames for 3-10 minutes, and caps at 100 frames for longer content, while enforcing a hard maximum extraction rate of 2 FPS.

The bradautomates/claude-video repository implements an adaptive frame extraction system that balances visual detail against token costs and processing time. Instead of using a fixed sampling rate, the frame budget scales dynamically based on the effective duration of the video or selected time window. This logic is centralized in skills/watch/scripts/frames.py and consumed by both the CLI tooling and the higher-level watch engine.

Understanding the Frame Budget Algorithm

The core scaling logic resides in the auto_fps function (lines 22-38 of skills/watch/scripts/frames.py). This function maps video duration to a target frame count, then calculates the appropriate frames-per-second (FPS) value while respecting safety caps.

Duration Tiers and Target Frames

The algorithm divides videos into five distinct duration buckets, each with a predefined target frame count:

• ≤ 30 seconds: target = min(max_frames, max(12, int(round(duration_seconds))))
• 30 seconds – 1 minute: target = 40
• 1 minute – 3 minutes: target = 60
• 3 minutes – 10 minutes: target = 80
• > 10 minutes: target = max_frames (default 100)

This tiered approach ensures that short clips receive dense sampling for detail, while long videos are capped to prevent excessive token usage.

The auto_fps Function Implementation

The primary scaling function calculates the effective duration from optional --start and --end arguments, then determines the appropriate budget:

def auto_fps(duration_seconds: float, max_frames: int = 100) -> tuple[float, int]:
    if duration_seconds <= 0:
        return 1.0, 1

    if duration_seconds <= 30:               # very short

        target = min(max_frames, max(12, int(round(duration_seconds))))
    elif duration_seconds <= 60:             # up to 1 min

        target = min(max_frames, 40)
    elif duration_seconds <= 180:            # up to 3 min

        target = min(max_frames, 60)
    elif duration_seconds <= 600:            # up to 10 min

        target = min(max_frames, 80)
    else:                                    # longer than 10 min

        target = max_frames

    return _clamp_fps(target / duration_seconds, duration_seconds, max_frames)

The function returns a tuple of (fps, target_frames), where the FPS value represents the actual extraction rate needed to achieve the target count across the video's duration.

Focused Mode Scaling with Time Windows

When users specify a focused range using --start and --end arguments, the system switches to auto_fps_focus. This variant applies a steeper scaling curve to the effective duration (the window between start and end times), yielding a higher frame density for the selected segment.

For example, a 30-second focused window might receive a 60-frame target instead of the standard 12-30 frames, ensuring richer detail in critical sections while maintaining the global MAX_FPS = 2.0 constraint.

FPS Capping and Safety Limits

Regardless of the calculated target, the _clamp_fps function enforces two critical safety constraints defined in skills/watch/scripts/config.py:

1. Maximum FPS ceiling: MAX_FPS = 2.0 prevents overly dense sampling
2. Absolute frame cap: The final count never exceeds the max_frames parameter (default 100)

def _clamp_fps(fps: float, duration_seconds: float, max_frames: int) -> tuple[float, int]:
    fps = min(fps, MAX_FPS)
    target = min(max_frames, max(1, int(round(fps * duration_seconds))))
    return fps, target

The actual frame extraction uses ffmpeg with both the computed FPS and a hard -frames:v limit, guaranteeing the budget is never exceeded:

cmd = [
    "ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
    "-i", str(Path(video_path).resolve()),
    "-vf", f"fps={fps},{_scale_filter(resolution)}",
    "-frames:v", str(max_frames), "-q:v", "4", output_pattern,
]

Practical Implementation Examples

CLI Extraction

Extract frames from a 45-second clip using default settings:

python -m skills.watch.scripts.frames \
    sample.mp4 frames_out \
    --max-frames 100

For this 45-second duration, the script selects target = 40, calculates fps = min(2.0, 40 / 45) ≈ 0.89, and extracts up to 40 JPEG frames.

Programmatic API Usage

Integrate the frame budget logic directly into Python applications:

from pathlib import Path
from skills.watch.scripts.frames import (
    get_metadata, auto_fps, extract, dedupe_perceptual,
)

video = Path("long_video.mp4")
meta = get_metadata(video)
duration = meta["duration_seconds"]

# Calculate budget for full video

fps, target = auto_fps(duration, max_frames=100)
print(f"Budget: {target} frames at {fps:.2f} FPS")

frames = extract(video, Path("out_dir"), fps=fps, max_frames=target)
frames, dropped = dedupe_perceptual(frames)
print(f"Extracted {len(frames)} frames, deduped {dropped}")

Focused Window Extraction

Calculate budget for a specific 30-second segment:

from skills.watch.scripts.frames import parse_time, auto_fps_focus

start = parse_time("02:00")      # 2 minutes

end = parse_time("02:30")        # 2 minutes 30 seconds

eff_dur = end - start            # 30 seconds

fps, target = auto_fps_focus(eff_dur, max_frames=100)
print(f"Focused window: {target} frames at {fps:.2f} FPS")

Summary

• Frame budget scales tiered: Videos receive 12-40 frames (0-1 min), 60 frames (1-3 min), 80 frames (3-10 min), or 100 frames (10+ min).
• Hard limits apply: All extraction respects MAX_FPS = 2.0 and the configurable max_frames cap (default 100).
• Focused windows dense: The auto_fps_focus function increases frame density for selected time ranges while maintaining safety caps.
• Implementation location: Core logic lives in skills/watch/scripts/frames.py, with configuration in skills/watch/scripts/config.py.

Frequently Asked Questions

What is the maximum number of frames Claude-Video will extract?

By default, the system caps extraction at 100 frames per video, configurable via the --max-frames argument or max_frames parameter. This limit applies regardless of video length, ensuring predictable token costs and processing times.

How does the frame budget change for short clips under 30 seconds?

Videos under 30 seconds receive a dynamic target calculated as max(12, round(duration_seconds)), meaning a 15-second clip gets 15 frames while a 5-second clip gets the minimum of 12 frames. This provides denser sampling for brief content where every frame matters.

What is the difference between auto_fps and auto_fps_focus?

auto_fps applies standard tiered scaling appropriate for full-video analysis, while auto_fps_focus uses an aggressive curve optimized for time-windowed segments (--start/--end). The focused variant allocates more frames per second of duration, providing richer detail for specific sections without exceeding global FPS or count limits.

Where is the frame budget logic configured in the codebase?

The primary implementation resides in skills/watch/scripts/frames.py, specifically in the auto_fps and _clamp_fps functions. Default constants like MAX_FPS = 2.0 are defined in skills/watch/scripts/config.py, and the logic is orchestrated by skills/watch/scripts/watch.py for end-to-end processing.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}