How Claude Video's Auto-FPS Logic Allocates Frames Based on Video Duration

Claude Video calculates an adaptive frame rate by dividing the video duration by a maximum frame budget of 200, clamping the result between 1 and 30 FPS, and passing this value to ffmpeg to ensure balanced temporal coverage without exceeding processing limits.

Claude Video is an open-source video processing framework that intelligently distributes frame extraction across clips of varying lengths. The auto-fps algorithm implemented in skills/watch/scripts/frames.py dynamically adjusts the extraction rate based on the input video's duration to balance detail capture against the MAX_FRAMES constraint defined in the configuration.

How the Auto-FPS Algorithm Works

Step 1: Retrieve Video Duration

The process begins in skills/watch/scripts/download.py, where ffprobe extracts metadata including the video length in seconds (duration_s). This value represents the total temporal span that requires frame coverage and serves as the primary input for the allocation calculation.

Step 2: Calculate the Ideal Frame Rate

In skills/watch/scripts/frames.py, the system computes a naive frames-per-second value by dividing the duration by the MAX_FRAMES constant (default 200). To prevent zero-frame scenarios for very short clips, the algorithm enforces a minimum of 1 FPS:

ideal_fps = max(1, math.floor(duration_s / MAX_FRAMES))

This calculation yields the rate required to distribute the frame budget evenly across the video timeline.

Step 3: Clamp to Valid Range

To prevent excessive extraction for long videos and ensure minimum coverage for short ones, the algorithm clamps ideal_fps between MIN_FPS (1) and MAX_FPS (30), as defined in skills/watch/scripts/config.py:

fps = max(MIN_FPS, min(ideal_fps, MAX_FPS))

Step 4: Execute Frame Extraction

The final fps value is passed to the ffmpeg command via the -vf fps={fps} filter. According to the source implementation, ffmpeg handles frame sampling at this specified rate, ensuring the output adheres to the calculated allocation while the MAX_FPS guard prevents system overload.

Frame Allocation Examples by Video Duration

The auto-fps logic produces different extraction densities depending on clip length:

• Short videos (≤ 200 seconds): For a 45-second clip, the calculation 45 / 200 = 0.225 floors to 0, then clamps to the MIN_FPS of 1. The ffmpeg command extracts 45 frames total (one per second).

• Medium videos (200–6000 seconds): A 5-minute clip (300 seconds) yields 300 / 200 = 1.5, clamped to 1 FPS, producing approximately 300 frames. A 10-minute clip (600 seconds) calculates to exactly 3 FPS, resulting in roughly 1,800 frames distributed across the timeline.

• Long videos (> 6000 seconds): For content exceeding approximately 100 minutes, the calculation duration_s / 200 exceeds 30 FPS, triggering the MAX_FPS ceiling. The extraction rate caps at 30 frames per second regardless of additional duration.

Core Implementation Details

The skills/watch/scripts/frames.py file contains the primary allocation logic, importing MAX_FRAMES, MIN_FPS, and MAX_FPS from the configuration module. The tests/test_frames.py suite validates these calculations across various duration inputs to ensure the frame budget constraints are respected.

# Example: 45-second clip allocation

# Duration = 45s, MAX_FRAMES = 200

# ideal_fps = max(1, floor(45 / 200)) = max(1, 0) = 1 FPS

# Result: 45 frames extracted (one per second)

# Example: 30-minute clip allocation  

# Duration = 1800s, MAX_FRAMES = 200

# ideal_fps = max(1, floor(1800 / 200)) = 9 FPS

# Result: ~16,200 frames before MAX_FPS guard limits

Summary

• Claude Video calculates ideal_fps by dividing video duration by the MAX_FRAMES budget (default 200) as implemented in skills/watch/scripts/frames.py.
• The algorithm enforces a minimum of 1 FPS (MIN_FPS) and maximum of 30 FPS (MAX_FPS) via configurable constants in skills/watch/scripts/config.py.
• Frame extraction uses ffmpeg with the -vf fps={fps} parameter to apply the calculated rate deterministically.
• Short videos receive dense 1 FPS coverage, while long videos scale up proportionally until hitting the 30 FPS ceiling.
• The implementation is validated by tests/test_frames.py across various video durations.

Frequently Asked Questions

How does Claude Video determine the video duration for FPS calculations?

Claude Video calls ffprobe within skills/watch/scripts/download.py to parse the video container metadata and extract the duration_s value. This represents the total length of the clip in seconds and feeds directly into the ideal_fps calculation in frames.py.

What is the maximum number of frames Claude Video will extract from a single video?

The default MAX_FRAMES constant is set to approximately 200 frames, defined in skills/watch/scripts/config.py. The auto-fps algorithm uses this value as the primary budget constraint when determining how frequently to sample frames from the input video.

Why does the auto-fps logic use a floor calculation instead of rounding?

The algorithm uses math.floor() in ideal_fps = max(1, math.floor(duration_s / MAX_FRAMES)) to ensure conservative frame allocation that stays within the budget. This approach only increases the extraction rate when the duration sufficiently exceeds the threshold, preventing premature frame exhaustion.

Can I customize the MIN_FPS and MAX_FPS constants in Claude Video?

Yes, these boundaries are defined as configurable constants in skills/watch/scripts/config.py. Modifying MIN_FPS and MAX_FPS allows you to adjust the extraction boundaries for specific use cases, such as requiring higher minimum density for short clips or raising the ceiling for high-motion analysis requiring more granular frames.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}