How Claude-Video Calculates Auto-FPS for Frame Extraction

Claude-Video dynamically derives extraction rates using an adaptive auto-FPS algorithm that samples short videos densely while capping long videos at a configurable frame budget, ensuring optimal token usage without manual tuning.

The bradautomates/claude-video repository implements an intelligent auto-FPS calculation mechanism for frame extraction that replaces fixed sampling rates with duration-aware logic. Instead of extracting frames at a constant rate regardless of video length, the system defined in skills/watch/scripts/frames.py calculates target frame counts based on video duration tiers. This approach preserves visual detail in brief clips while preventing context window overflow from extended content.

Core Auto-FPS Logic in frames.py

The auto-FPS system resides in skills/watch/scripts/frames.py and consists of two primary calculation functions and a safety enforcement helper. These work together to respect the max_frames budget (defaulting to 100) while adapting to whether the user scans the full video or a focused segment.

Full-Video Scans with auto_fps()

When processing complete videos without user-specified start or end times, the auto_fps(duration_seconds, max_frames=100) function applies a duration-based tiering system to determine the target frame count:

• ≤ 30 seconds: max(12, round(duration)) frames
• ≤ 60 seconds: 40 frames
• ≤ 180 seconds: 60 frames
• ≤ 600 seconds: 80 frames
• > 600 seconds: max_frames (default 100)

After calculating the target, the function passes the values to _clamp_fps() to derive the final FPS and ensure the frame count does not exceed the budget.

Focused Ranges with auto_fps_focus()

For extracted segments specified via --start or --end flags, the auto_fps_focus(duration_seconds, max_frames=100) function applies tighter sampling budgets appropriate for shorter analysis windows:

• ≤ 5 seconds: Up to 6× duration (capped)
• ≤ 15 seconds: Up to 4× duration
• ≤ 30 seconds: 60 frames
• ≤ 60 seconds: 80 frames
• ≤ 180 seconds: max_frames
• > 180 seconds: max_frames

This function also routes its output through _clamp_fps() to enforce safety limits.

Safety Limits via _clamp_fps()

The _clamp_fps() helper enforces two critical constraints defined in the source code:

def _clamp_fps(fps: float, duration_seconds: float, max_frames: int) -> tuple[float, int]:
    fps = min(fps, MAX_FPS)                                   # never exceed 2 fps

    target = min(max_frames, max(1, int(round(fps * duration_seconds))))
    return fps, target

The MAX_FPS constant (set to 2.0) prevents overly aggressive extraction that would overwhelm downstream token limits. The target calculation guarantees at least one frame while capping the total to max_frames, returning a tuple of (fps, target) for downstream consumption.

Integration with the Watch Pipeline

In skills/watch/scripts/watch.py, the orchestration logic determines which auto-FPS function to invoke based on user input:

if focused:
    fps, target = auto_fps_focus(effective_duration, max_frames=budget_cap)
else:
    fps, target = auto_fps(effective_duration, max_frames=budget_cap)

When users supply an explicit --fps flag, that value overrides the automatic calculation but still respects the safety clamp:

if args.fps is not None:
    fps = min(args.fps, MAX_FPS)
    target = max(1, int(round(fps * effective_duration)))

This ensures that even manual FPS specifications cannot exceed the 2 FPS hard limit or the frame budget.

Practical Implementation Examples

The auto-FPS mechanism can be observed directly in Python:

>>> from skills.watch.scripts.frames import auto_fps, auto_fps_focus

# Full-video scan of a 45-second clip, default max 100 frames

>>> auto_fps(45)
(0.89, 40)          # 0.89 fps → 40 frames (capped by budget)

# Focused window of 8 seconds, same max-frame budget

>>> auto_fps_focus(8)
(2.0, 16)           # Clamped to MAX_FPS = 2 fps → 16 frames

# Override with a user-provided FPS of 1.5 fps for a 120-second video

>>> from skills.watch.scripts.frames import _clamp_fps
>>> _clamp_fps(1.5, 120, max_frames=100)
(1.5, 100)          # 1.5 fps gives 180 frames, but capped at 100

When invoking the CLI, the calculated target appears in the extraction logs:

$ python -m skills.watch.scripts.watch "https://youtu.be/xyz" --detail balanced
[watch] extracting scene-aware frames over full 180.0s (target 80, cap 80) …

The target value printed above originates from the auto_fps() function's duration-based tiering.

Summary

• skills/watch/scripts/frames.py contains the core auto_fps() and auto_fps_focus() functions that calculate target frame counts based on video duration tiers.
• _clamp_fps() enforces a maximum extraction rate of 2 FPS and ensures the final frame count stays within the user-defined or default budget.
• Full-video scans use auto_fps() with tiered targets (12-100 frames depending on length), while focused ranges use auto_fps_focus() with denser sampling for short windows.
• skills/watch/scripts/watch.py orchestrates the selection between full-scan and focused logic, applying user FPS overrides when specified while maintaining safety limits.
• The system guarantees at least one extracted frame per video while preventing context window overflow through hard caps on both frame rate and total frame count.

Frequently Asked Questions

What is the maximum FPS cap in Claude-Video's auto-FPS system?

The MAX_FPS constant in skills/watch/scripts/frames.py sets a hard limit of 2.0 FPS. This ceiling prevents excessive frame extraction that could overwhelm downstream language model token limits, and it applies to both automatically calculated rates and user-specified --fps overrides.

How does the auto-FPS calculation differ between full videos and focused ranges?

Full-video scans use auto_fps(), which applies conservative tiering (e.g., 40 frames for 60-second videos), while focused ranges use auto_fps_focus(), which allocates denser sampling for short segments (e.g., up to 6× duration for clips under 5 seconds). The focused function assumes users analyzing specific timestamps require higher temporal resolution.

Can users override the automatic frame rate calculation?

Yes. Users can specify a fixed FPS using the --fps flag in the CLI. However, the system still passes this value through _clamp_fps(), ensuring it does not exceed 2.0 FPS and that the resulting frame count respects the max_frames budget. If no FPS is specified, the system automatically selects the appropriate rate based on video duration.

Where is the frame extraction logic implemented in the codebase?

The auto-FPS calculation logic lives in skills/watch/scripts/frames.py, specifically within the auto_fps(), auto_fps_focus(), and _clamp_fps() functions. The orchestration logic that decides which function to call and handles user overrides resides in skills/watch/scripts/watch.py.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}