How Frames Are Scaled and FPS-Clamped for Claude Read Compatibility

The watch skill in bradautomates/claude-video ensures Claude Read compatibility by applying a FFmpeg scale filter that limits dimensions to 1998×512px while preserving aspect ratio, and clamping frame rates to a hard-coded maximum of 2 FPS to manage token budgets.

The bradautomates/claude-video repository implements a robust frame extraction pipeline that guarantees every image meets Anthropic's strict dimensional and rate limits for AI consumption. By combining intelligent resolution scaling with aggressive FPS clamping, the watch skill prevents token budget overflow while maintaining sufficient visual fidelity for video analysis tasks.

Resolution Scaling with FFmpeg

The core scaling logic resides in skills/watch/scripts/frames.py, specifically within the _scale_filter() function (lines 42-55). This utility generates an FFmpeg filter string that enforces hard dimensional boundaries required for Claude Read compatibility.

The _scale_filter() Implementation

The _scale_filter() function dynamically constructs a scale filter that caps width to the user-requested resolution (defaulting to 512px) while ensuring height never exceeds the MAX_READ_DIMENSION constant of 1998px:

def _scale_filter(resolution: int) -> str:
    return (
        f"scale=w='min({resolution},iw)':h='min({MAX_READ_DIMENSION},ih)':"
        "force_original_aspect_ratio=decrease:force_divisible_by=2"
    )

This filter string is injected into every FFmpeg command via the -vf flag. For example, in the extract() method at line 94, the command includes "-vf", f"fps={fps},{_scale_filter(resolution)}", ensuring scaling applies uniformly across all extraction modes including extract_scene_candidates() and extract_keyframes().

Aspect Ratio and Dimension Constraints

The scale filter employs four critical parameters to ensure Claude Read compatibility:

• w='min({resolution},iw)' - Caps width to the requested resolution or preserves original width if smaller
• h='min({MAX_READ_DIMENSION},ih)' - Hard-limits height to 1998px regardless of source material
• force_original_aspect_ratio=decrease - Maintains aspect ratio by only decreasing dimensions, never stretching
• force_divisible_by=2 - Ensures even pixel dimensions for codec compatibility

FPS Clamping for Token Budget Management

Beyond spatial constraints, the repository implements strict temporal limits through the _clamp_fps() function to prevent excessive frame generation that would deplete Claude's token budget.

The _clamp_fps() Function Logic

Located in skills/watch/scripts/frames.py, this function enforces the MAX_FPS = 2.0 ceiling while calculating optimal frame counts:

def _clamp_fps(fps: float, duration_seconds: float, max_frames: int) -> tuple[float, int]:
    fps = min(fps, MAX_FPS)
    target = min(max_frames, max(1, int(round(fps * duration_seconds))))
    return fps, target

The function first clamps the requested FPS to the 2.0 maximum, then calculates a target frame count based on the clamped rate and video duration. Both auto_fps() and auto_fps_focus() invoke this guard before extraction begins, ensuring the pipeline never generates more than two frames per second regardless of user input.

End-to-End Frame Extraction Pipeline

When processing a video via the /watch slash command, the orchestration in skills/watch/scripts/watch.py (lines 30-38) coordinates these safety mechanisms:

1. Metadata extraction - get_metadata() retrieves duration_seconds from the source video
2. FPS determination - auto_fps() or auto_fps_focus() selects a rate, then _clamp_fps() caps it at 2 FPS
3. Frame extraction - extract() generates an FFmpeg command combining the clamped FPS with the scale filter

The resulting JPEG outputs conform to strict Claude Read compatibility standards: maximum 512px width (configurable), maximum 1998px height, even dimensions, and no more than 2 frames per second.

Practical Implementation Examples

Generating the scale filter string:

from pathlib import Path
from skills.watch.scripts.frames import _scale_filter

resolution = 800
filter_str = _scale_filter(resolution)
print(filter_str)

# Output: scale=w='min(800,iw)':h='min(1998,ih)':force_original_aspect_ratio=decrease:force_divisible_by=2

Clamping FPS for a 45-second clip:

from skills.watch.scripts.frames import _clamp_fps

fps_requested = 5.0          # User request exceeds limits

duration = 45.0           
max_frames = 150

fps, target = _clamp_fps(fps_requested, duration, max_frames)
print(fps, target)          # Result: 2.0 90

Complete FFmpeg command structure:

ffmpeg -hide_banner -loglevel error -y \
  -i input.mp4 \
  -vf "fps=2.0,scale=w='min(512,iw)':h='min(1998,ih)':force_original_aspect_ratio=decrease:force_divisible_by=2" \
  -frames:v 100 -q:v 4 out/frame_%04d.jpg

Summary

• The watch skill guarantees Claude Read compatibility by enforcing dimensional limits of 1998px height and configurable width (default 512px) through the _scale_filter() function in skills/watch/scripts/frames.py.
• FPS clamping prevents token budget overflow by hard-limiting extraction rates to 2.0 FPS via _clamp_fps(), regardless of user requests or source video frame rates.
• The FFmpeg scale filter preserves original aspect ratios while ensuring even pixel dimensions through force_original_aspect_ratio=decrease and force_divisible_by=2 parameters.
• All extraction methods including extract(), extract_scene_candidates(), and extract_keyframes() automatically apply these constraints through the centralized filter generation functions.

Frequently Asked Questions

What is the maximum frame rate allowed for Claude Read compatibility?

The hard-coded limit is 2.0 FPS as defined by the MAX_FPS constant in skills/watch/scripts/frames.py. Even if users request higher rates through the interface, the _clamp_fps() function automatically reduces the value to 2.0 FPS to prevent excessive token consumption during Claude's analysis phase.

Why is the maximum height limited to 1998 pixels?

The MAX_READ_DIMENSION constant of 1998px represents Anthropic's documented maximum read dimension for image inputs. This limit ensures that extracted frames remain within Claude's context window constraints while providing sufficient vertical resolution for detecting text, objects, and scene changes in video content.

How does the watch skill preserve aspect ratio when scaling frames?

The _scale_filter() function includes force_original_aspect_ratio=decrease in its FFmpeg filter string, which instructs the encoder to scale down using the smaller of the two dimensional ratios. This guarantees that neither width nor height exceeds its respective limit while maintaining the original video's proportional geometry without stretching or distortion.

What happens if I request a frame rate higher than 2 FPS?

The system silently clamps your request to 2.0 FPS through the _clamp_fps() function. For example, requesting 5 FPS on a 45-second video returns a clamped rate of 2.0 FPS and a target frame count of 90 frames (2 × 45) rather than the 225 frames that 5 FPS would generate. This ensures predictable token costs regardless of user input.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}