How the Frame Budget System in Claude-Video Scales With Video Duration

The frame budget system in Claude-Video uses a dynamic FPS calculation that automatically adjusts the extraction rate based on video length, ensuring the total frame count never exceeds a hard limit regardless of source duration.

Claude-Video implements a budget-aware extraction algorithm that guarantees predictable processing overhead for videos of any length. As implemented in bradautomates/claude-video, the system scales frame sampling inversely with video duration to maintain a fixed maximum workload. This approach ensures that both 10-second clips and 10-minute documentaries receive representative visual coverage without overwhelming downstream AI processing steps.

How the Frame Budget Algorithm Works

The scaling mechanism relies on three core components that work together to bound computational cost while preserving visual information.

The Hard Cap on Frame Count (MAX_FRAMES)

In skills/watch/scripts/config.py, the system defines a hard upper limit (MAX_FRAMES, default approximately 300) that caps the total number of frames ever generated from a single video. This constant acts as the primary constraint that drives all subsequent calculations. No matter how long the input video, the extractor will never produce more than this specified number of frames.

Dynamic FPS Calculation Based on Duration

Before extraction begins, the download script in skills/watch/scripts/download.py invokes ffprobe to obtain the video's exact duration in seconds (duration_secs). The system then computes a target frames-per-second (target_fps) using the formula:

[ \text{target_fps} = \min(\text{DEFAULT_FPS},; \frac{\text{MAX_FRAMES}}{\text{duration_secs}}) ]

• For short videos (e.g., 10 seconds), the division MAX_FRAMES / duration_secs yields a value higher than DEFAULT_FPS (typically 30 fps), so the extractor runs at the default rate.
• For longer videos (e.g., 5 minutes = 300 seconds), the calculation drops to ≤ 1 fps, forcing the extractor to sample fewer frames per second while keeping the total count under MAX_FRAMES.

Implementation in the Source Code

The budget logic is distributed across three specialized modules that handle metadata detection, mathematical scaling, and physical extraction.

Duration Detection with ffprobe

The download.py script retrieves video metadata using ffprobe to determine the exact duration before any frame processing occurs. This measurement feeds directly into the fps calculation, ensuring the budget constraint is applied before expensive extraction begins.

Frame Extraction in frames.py

The skills/watch/scripts/frames.py module implements the actual extraction logic. It passes the computed target_fps to ffmpeg using the -vf fps={target_fps} filter. Because ffmpeg's fps filter transparently drops or duplicates frames to match the target rate, the final output strictly adheres to the calculated budget. The module executes a command structure similar to:

# Conceptual implementation from frames.py

MAX_FRAMES = 300               # Hard cap defined in config.py

DEFAULT_FPS = 30               # Preferred extraction rate

duration_secs = get_video_duration(path)   # ffprobe call from download.py

target_fps = min(DEFAULT_FPS, MAX_FRAMES / duration_secs)

# ffmpeg execution with dynamic fps filter

ffmpeg -i INPUT -vf fps={target_fps} -q:v 2 frames/%05d.jpg

Practical Examples

The scaling behavior manifests differently depending on input duration. Here is how the system processes varying video lengths:

# Example: Processing a 15-second short clip

duration = 15
target_fps = min(30, 300 / 15)  # Returns 20 fps (less than default 30)

# Result: ~300 frames extracted at full video rate

# Example: Processing a 10-minute documentary

duration = 600
target_fps = min(30, 300 / 600)  # Returns 0.5 fps

# Result: Exactly 300 frames extracted, one every 2 seconds

For manual implementation mirroring the skill's behavior:

import subprocess
import json
import pathlib

def get_duration(path: str) -> float:
    """Obtain video duration using ffprobe (as implemented in download.py)"""
    out = subprocess.check_output([
        "ffprobe", "-v", "error", "-show_entries",
        "format=duration", "-of", "json", path
    ])
    return float(json.loads(out)["format"]["duration"])

def extract_frames(video_path: str, out_dir: str, max_frames: int = 300, default_fps: int = 30):
    """Budget-aware frame extraction matching frames.py logic"""
    dur = get_duration(video_path)
    target_fps = min(default_fps, max_frames / dur)
    
    pathlib.Path(out_dir).mkdir(parents=True, exist_ok=True)
    subprocess.run([
        "ffmpeg", "-i", video_path,
        "-vf", f"fps={target_fps}",
        "-q:v", "2",
        f"{out_dir}/%05d.jpg"
    ], check=True)

Summary

• Hard ceiling: The MAX_FRAMES constant (default ~300) in skills/watch/scripts/config.py establishes an absolute upper bound on extracted frames.
• Inverse scaling: The system calculates target_fps as min(DEFAULT_FPS, MAX_FRAMES / duration), automatically reducing sampling rates for longer videos.
• Linear time: Frame extraction time grows roughly linearly with video duration, but the computational load for downstream processing remains constant.
• Implementation: skills/watch/scripts/download.py handles duration detection, while skills/watch/scripts/frames.py executes the budget-compliant ffmpeg extraction.

Frequently Asked Questions

What is the maximum number of frames Claude-Video extracts?

According to the source code in skills/watch/scripts/config.py, the default MAX_FRAMES constant is set to approximately 300 frames. This hard limit applies universally regardless of input video length or resolution.

How does the frame budget system handle very short videos?

For videos where MAX_FRAMES / duration_secs exceeds DEFAULT_FPS (typically 30 fps), the system uses the default fps value instead. This means short clips under 10 seconds are processed at full frame rate, capturing all available visual information without hitting the budget cap.

Does the frame budget affect video processing time?

While the frame extraction phase takes longer for lengthier videos (due to reading more source material), the budget system ensures that downstream AI processing steps receive a consistent ~300 frames maximum. This prevents computational costs from scaling uncontrollably with input duration.

Where are the frame budget constants configured?

The primary configuration values MAX_FRAMES and DEFAULT_FPS reside in skills/watch/scripts/config.py. The calculation logic using these constants appears in skills/watch/scripts/frames.py, which references duration data obtained by skills/watch/scripts/download.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}