How Frame Budget Scales with Video Duration in Claude-Video

The frame budget scales non-linearly with video duration using a tiered algorithm that extracts 12–40 frames for clips under one minute, 60 frames for 1–3 minutes, 80 frames for 3–10 minutes, and caps at 100 frames for longer videos while enforcing a hard limit of 2 FPS.

Understanding how the frame budget scale with video duration works is essential for predicting token usage and processing time in the bradautomates/claude-video repository. The system implements a budget-by-duration strategy in skills/watch/scripts/frames.py that balances visual detail against computational cost, automatically adjusting extraction density based on content length.

Duration-Based Frame Budget Algorithm

The core scaling logic resides in the auto_fps function within skills/watch/scripts/frames.py. This function maps video duration to a target frame count using tiered thresholds, then calculates the appropriate frames-per-second (FPS) rate while respecting safety caps.

Calculating Effective Duration

Before applying the budget tiers, the system determines the effective duration by parsing optional --start and --end arguments or defaulting to the full video length:

meta = get_metadata(video)
start_sec = parse_time(start_arg)
end_sec   = parse_time(end_arg)
effective_duration = max(0.0,
    (end_sec if end_sec is not None else meta["duration_seconds"])
    - (start_sec if start_sec is not None else 0.0))

The auto_fps Scaling Function

The auto_fps function (lines 22–38 of skills/watch/scripts/frames.py) implements the tiered scaling logic:

def auto_fps(duration_seconds: float, max_frames: int = 100) -> tuple[float, int]:
    if duration_seconds <= 0:
        return 1.0, 1

    if duration_seconds <= 30:               # very short

        target = min(max_frames, max(12, int(round(duration_seconds))))
    elif duration_seconds <= 60:             # up to 1 min

        target = min(max_frames, 40)
    elif duration_seconds <= 180:            # up to 3 min

        target = min(max_frames, 60)
    elif duration_seconds <= 600:            # up to 10 min

        target = min(max_frames, 80)
    else:                                    # longer than 10 min

        target = max_frames

    return _clamp_fps(target / duration_seconds, duration_seconds, max_frames)

FPS Clamping and Safety Limits

The _clamp_fps helper enforces the MAX_FPS = 2.0 ceiling (defined in the configuration) and ensures the final frame count never exceeds the max_frames parameter:

def _clamp_fps(fps: float, duration_seconds: float, max_frames: int) -> tuple[float, int]:
    fps = min(fps, MAX_FPS)
    target = min(max_frames, max(1, int(round(fps * duration_seconds))))
    return fps, target

Frame Budget Scaling Tiers

The following table summarizes how the frame budget scale with video duration operates under default settings (max_frames=100):

Video Duration	Target Frames	Effective FPS Calculation
≤ 30 seconds	max(12, round(duration))	target / duration (capped at 2.0)
30 sec – 1 min	40 frames	≤ 0.67 FPS
1 min – 3 min	60 frames	≤ 1.0 FPS
3 min – 10 min	80 frames	≤ 1.33 FPS
> 10 minutes	100 frames (hard cap)	100 / duration (often < 0.17 FPS)

For a 45-second clip, the algorithm selects a target of 40 frames, computes fps = min(2.0, 40/45) ≈ 0.89, and extracts frames at approximately 0.89 FPS.

Focused Window Mode for Dense Sampling

When users specify a time window using --start and --end arguments, the system switches to auto_fps_focus. This variant applies a steeper scaling curve to the effective duration, allocating a higher frame budget for the same time window to ensure richer detail in the selected segment.

While the focused mode increases the target frame count for short durations, it still respects the global MAX_FPS = 2.0 limit and the max_frames cap defined in the configuration.

Practical Implementation Examples

Command-Line Extraction

Extract frames using the default budget scaling:

$ python -m skills.watch.scripts.frames \
    sample.mp4 frames_out \
    --max-frames 100

For a 45-second video, this command generates approximately 40 JPEG frames at 0.89 FPS.

Programmatic Budget Calculation

Access the scaling logic directly in Python:

from pathlib import Path
from skills.watch.scripts.frames import (
    get_metadata, auto_fps, extract, dedupe_perceptual,
)

video = Path("long_video.mp4")
meta = get_metadata(video)
duration = meta["duration_seconds"]

# Calculate budget for full video

fps, target = auto_fps(duration, max_frames=100)
print(f"Budget: {target} frames → {fps:.2f} FPS")

frames = extract(video, Path("out_dir"), fps=fps, max_frames=target)
frames, dropped = dedupe_perceptual(frames)
print(f"Extracted {len(frames)} frames, deduped {dropped}")

Focused Window Extraction

For a specific 30-second segment starting at 2 minutes:

from skills.watch.scripts.frames import parse_time, auto_fps_focus

start = parse_time("02:00")      # 2 minutes

end   = parse_time("02:30")      # 2 minutes 30 seconds

eff_dur = end - start            # 30 seconds

fps, target = auto_fps_focus(eff_dur, max_frames=100)
print(f"Focused window budget: {target} frames → {fps:.2f} FPS")

This focused approach yields a denser sampling (up to 60 frames for a 30-second window) compared to the standard algorithm, while maintaining the 2 FPS ceiling.

Summary

• The frame budget scale with video duration follows a tiered allocation: 12–40 frames for sub-minute content, 60 frames for 1–3 minutes, 80 frames for 3–10 minutes, and a 100-frame cap beyond 10 minutes.
• The auto_fps function in skills/watch/scripts/frames.py implements this logic, while _clamp_fps enforces the MAX_FPS = 2.0 safety limit.
• Focused mode (auto_fps_focus) increases frame density for time-windowed segments but respects the same global caps.
• The final extraction uses ffmpeg with -frames:v set to the calculated budget, guaranteeing hard limits on output quantity.

Frequently Asked Questions

What is the maximum frame budget for long videos?

Videos longer than 10 minutes are capped at 100 frames by default (configurable via the max_frames parameter). According to the source code in skills/watch/scripts/frames.py, the else branch of auto_fps sets target = max_frames for any duration exceeding 600 seconds, ensuring token costs remain predictable regardless of video length.

How does Claude-Video handle very short clips under 30 seconds?

For clips ≤ 30 seconds, the algorithm uses max(12, round(duration_seconds)) to ensure a minimum viable sample of 12 frames while avoiding over-sampling. This means a 10-second video receives 12 frames (1.2 FPS), while a 25-second video receives 25 frames (1.0 FPS), all subject to the 2 FPS maximum enforced by _clamp_fps.

What is the difference between auto_fps and auto_fps_focus?

auto_fps applies the standard tiered scaling suitable for full-video analysis, while auto_fps_focus uses an aggressive curve optimized for partial segments specified by --start and --end arguments. The focused variant allocates more frames per second of duration to provide richer detail in zoomed-in windows, though both functions respect the MAX_FPS = 2.0 and max_frames constraints defined in the configuration.

Where is the frame budget logic implemented in the codebase?

The primary implementation resides in skills/watch/scripts/frames.py, specifically in the auto_fps (lines 22–38), auto_fps_focus, and _clamp_fps functions. Default constants like MAX_FPS are defined in skills/watch/scripts/config.py, while the orchestration layer in skills/watch/scripts/watch.py coordinates the end-to-end extraction pipeline.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}