How Claude-Video Handles Frame Budgeting Based on Video Duration

Claude-Video extracts video frames with a budget-aware algorithm that guarantees a predictable amount of work regardless of how long the source video is.

The bradautomates/claude-video repository implements an intelligent frame extraction system that prevents processing overload by dynamically adjusting sampling rates inversely with video length. This approach ensures consistent computational costs and predictable downstream processing whether analyzing short clips or feature-length content.

The Core Budgeting Strategy

Claude-Video's frame budgeting relies on three fundamental components working together to cap resource usage while maintaining visual representativeness.

Hard Frame Limits

At the center of the system is MAX_FRAMES, a configuration constant defined in skills/watch/scripts/config.py that sets a hard upper limit of approximately 300 frames. This cap ensures that downstream AI processing steps—such as captioning or transcription—receive a manageable, bounded dataset regardless of input video length.

Duration Detection

Before extraction begins, the download script in skills/watch/scripts/download.py invokes ffprobe to obtain the video's exact duration in seconds (duration_secs). This metadata retrieval step is essential for calculating the appropriate sampling rate that will keep the final frame count under the MAX_FRAMES threshold.

Dynamic FPS Calculation

The frames.py script computes a target frames-per-second value using the formula:

target_fps = min(DEFAULT_FPS, MAX_FRAMES / duration_secs)

Where DEFAULT_FPS (typically 30 fps) represents the preferred extraction rate for short content. For videos where MAX_FRAMES / duration_secs exceeds DEFAULT_FPS, the system extracts at the default rate. For longer videos, the calculation yields a lower fps, forcing the extractor to sample fewer frames per second while keeping the total count under budget.

Source Code Implementation

The budgeting logic spans three critical files in the skills/watch module:

• skills/watch/scripts/config.py – Stores default configuration values including MAX_FRAMES and DEFAULT_FPS
• skills/watch/scripts/download.py – Retrieves video metadata including duration via ffprobe
• skills/watch/scripts/frames.py – Implements the budget-aware fps calculation and executes ffmpeg

FFmpeg Integration

The computed target_fps value passes directly to ffmpeg's fps filter:

ffmpeg -i INPUT -vf fps={target_fps} -q:v 2 frames/%05d.jpg

Because ffmpeg's fps filter transparently drops or duplicates frames to match the target rate, the final output never exceeds the configured maximum.

Practical Code Examples

The following examples demonstrate the frame budgeting behavior for different video durations.

Short Video Handling (High FPS)

For a 10-second video, the calculation 300 / 10 = 30 matches DEFAULT_FPS, resulting in full-rate extraction:

MAX_FRAMES = 300
DEFAULT_FPS = 30
duration_secs = 10.0
target_fps = min(30, 300 / 10)  # Returns 30 fps

# Total frames: 300 (at 30 fps for 10s)

Long Video Handling (Reduced FPS)

For a 5-minute (300-second) video, the system automatically throttles the extraction rate:

duration_secs = 300.0
target_fps = min(30, 300 / 300)  # Returns 1.0 fps

# Total frames: 300 (at 1 fps for 300s)

Complete Extraction Script

This Python implementation mirrors the skill's internal behavior:

import subprocess
import json
import pathlib

def get_duration(path):
    """Retrieve video duration using ffprobe (as implemented in download.py)"""
    out = subprocess.check_output([
        "ffprobe", "-v", "error", "-show_entries",
        "format=duration", "-of", "json", path
    ])
    return float(json.loads(out)["format"]["duration"])

def extract_frames(video_path, out_dir, max_frames=300, default_fps=30):
    """Extract frames with budget-aware fps calculation (frames.py logic)"""
    dur = get_duration(video_path)
    target_fps = min(default_fps, max_frames / dur)
    
    pathlib.Path(out_dir).mkdir(parents=True, exist_ok=True)
    subprocess.run([
        "ffmpeg", "-i", video_path,
        "-vf", f"fps={target_fps}",
        "-q:v", "2",
        f"{out_dir}/%05d.jpg"
    ], check=True)

Summary

• Hard cap: Claude-Video enforces a MAX_FRAMES limit (default ~300) via skills/watch/scripts/config.py to bound computational costs
• Dynamic adjustment: The system calculates target_fps = min(DEFAULT_FPS, MAX_FRAMES / duration_secs) in skills/watch/scripts/frames.py
• Duration detection: Video length is obtained via ffprobe in skills/watch/scripts/download.py before extraction begins
• Linear scaling: Frame extraction time grows roughly linearly with video duration, but the frame count remains constant
• FFmpeg integration: The fps filter receives the computed target rate to ensure the budget is never exceeded

Frequently Asked Questions

What is the default maximum frame limit in Claude-Video?

Claude-Video defaults to approximately 300 frames as defined by the MAX_FRAMES constant in skills/watch/scripts/config.py. This hard limit ensures that downstream processing steps receive a predictable, bounded dataset regardless of whether the input is a 10-second clip or a 2-hour film.

How does Claude-Video handle videos shorter than 10 seconds?

For short videos where MAX_FRAMES / duration_secs exceeds DEFAULT_FPS (typically 30 fps), the system extracts frames at the full default rate. A 10-second video generates approximately 300 frames at 30 fps, utilizing the entire budget to maximize visual detail for brief content.

Which component detects the video duration before frame extraction?

The download.py script in skills/watch/scripts/ calls ffprobe with the -show_entries format=duration flag to retrieve precise video length in seconds. This duration value feeds into the fps calculation performed by frames.py to determine the appropriate sampling rate.

What ffmpeg filter performs the actual frame rate adjustment?

Claude-Video passes the computed target_fps value to ffmpeg's fps video filter (-vf fps={target_fps}). This filter automatically drops or duplicates frames as needed to maintain the exact target rate, guaranteeing the final frame count never exceeds the configured MAX_FRAMES budget.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}