How Claude-Video’s Frame Budget Scales With Video Duration for Different Time Ranges

Claude-Video employs a budget-aware algorithm that dynamically adjusts the extraction frame rate based on video length, ensuring the total frame count never exceeds a hard cap regardless of source duration.

The open-source bradautomates/claude-video repository implements a predictable scaling strategy to prevent frame extraction from overwhelming downstream AI processing pipelines. Unlike fixed frame-rate extraction that generates linearly more frames as duration increases, the system automatically throttles its sampling rate to maintain a consistent computational budget across videos ranging from seconds to hours.

Understanding the Frame Budget Algorithm

The frame extraction logic revolves around two primary constants and a duration-aware calculation defined across the skill’s script files.

The Hard Cap: MAX_FRAMES

In skills/watch/scripts/config.py, the system defines a hard upper limit (MAX_FRAMES, default approximately 300) that acts as an absolute ceiling on frame generation. This cap ensures that memory usage, storage requirements, and processing time remain bounded even when handling multi-hour source files.

Duration Detection via ffprobe

Before extraction begins, skills/watch/scripts/download.py invokes ffprobe to retrieve the video’s exact duration in seconds (duration_secs). This metadata serves as the input variable that drives the dynamic scaling calculation.

Dynamic FPS Calculation for Different Time Ranges

The scaling mechanism lives in skills/watch/scripts/frames.py, where the system computes a target frame rate using the formula:

[ \text{target_fps} = \min(\text{DEFAULT_FPS},; \frac{\text{MAX_FRAMES}}{\text{duration_secs}}) ]

This formula creates distinct behavioral zones based on video length:

Short Videos (Under 10 Seconds)

For brief clips (e.g., 10 seconds), the calculation yields 30 fps (300/10), which equals or exceeds DEFAULT_FPS. The min() function caps extraction at the default rate, ensuring high-fidelity sampling without generating redundant frames beyond the hardware’s intended capture rate.

Medium Videos (10 Seconds to 5 Minutes)

As duration increases to 5 minutes (300 seconds), the division drops to ≤1 fps. The system automatically transitions from the default 30 fps to the calculated rate, sampling one frame per second to maintain the budget. This represents the inflection point where the frame budget begins to constrain the extraction density.

Long Videos (Over 5 Minutes)

For extended content, the frame rate continues dropping proportionally. A 10-minute video (600 seconds) extracts at 0.5 fps, while hour-long content may sample as infrequently as once every 12 seconds. The linear scaling ensures the total output never exceeds 300 frames regardless of input length.

Implementation in the Source Code

The frames.py script passes the computed target_fps directly to ffmpeg using the fps video filter (-vf fps={target_fps}). Because ffmpeg’s filter transparently drops frames when downsampling, the resulting output directory contains exactly the budgeted amount of evenly distributed representative frames.

Practical Code Example

To manually extract frames using the same budget-aware logic found in the repository:

import subprocess
import json
import pathlib

def get_duration(video_path):
    """Extract duration using ffprobe as implemented in download.py"""
    cmd = [
        "ffprobe", "-v", "error",
        "-show_entries", "format=duration",
        "-of", "json", video_path
    ]
    output = subprocess.check_output(cmd)
    return float(json.loads(output)["format"]["duration"])

def extract_frames(video_path, output_dir, max_frames=300, default_fps=30):
    """Mirror the budget-aware extraction from frames.py"""
    duration = get_duration(video_path)
    target_fps = min(default_fps, max_frames / duration)
    
    pathlib.Path(output_dir).mkdir(parents=True, exist_ok=True)
    
    subprocess.run([
        "ffmpeg", "-i", video_path,
        "-vf", f"fps={target_fps}",
        "-q:v", "2",
        f"{output_dir}/%05d.jpg"
    ], check=True)

# Command-line usage leveraging the automatic scaling

$ claudecode /watch https://example.com/long-video.mp4

# Automatically computes ~1 fps for 5-minute content vs 30 fps for 10-second content

Summary

• Hard limit: The system enforces a MAX_FRAMES cap (default ~300) defined in skills/watch/scripts/config.py to bound computational load.
• Dynamic calculation: target_fps is calculated as min(DEFAULT_FPS, MAX_FRAMES / duration_secs) in skills/watch/scripts/frames.py.
• Automatic throttling: Short videos extract at full 30 fps, while longer videos automatically downsample to fractional rates.
• Consistent budget: Frame extraction time grows linearly with duration, but the output frame count remains constant regardless of input length.

Frequently Asked Questions

How does claude-video handle videos longer than 10 minutes?

For videos exceeding 10 minutes (600 seconds), the calculated target_fps drops below 0.5 frames per second according to the formula in skills/watch/scripts/frames.py. The system continues dividing 300 by the duration, potentially extracting only one frame every several seconds while maintaining the hard cap, ensuring processing remains feasible for hour-long content.

Does the frame budget affect video quality or analysis accuracy?

The budget prioritizes temporal coverage over frame density. While long videos sample fewer frames per second, the evenly distributed extraction across the entire duration preserves representative visual context for AI analysis, preventing the system from processing only the opening sequence of long-form content.

Where can I modify the MAX_FRAMES limit?

The constant resides in skills/watch/scripts/config.py alongside DEFAULT_FPS. Adjusting MAX_FRAMES requires restarting the skill, as the values load at initialization and propagate to frames.py during the extraction pipeline setup.

Why use min() instead of always calculating MAX_FRAMES divided by duration?

The min() guard preserves high-fidelity extraction for brief clips where generating 300 frames at 30 fps is possible (clips under 10 seconds). Without this protection, the formula would artificially inflate frame rates beyond 30 fps for very short videos, creating redundant data and unnecessary storage overhead while providing no analytical benefit.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}