How Claude Video Implements Frame Cap Logic for Different Detail Modes

Claude Video caps the number of frames extracted from a video based on the detail argument, with limits of 50 frames for "efficient", 100 frames for "balanced", and no limit for "token-burner" or "transcript" modes, defaulting to 100 for unknown values.

The bradautomates/claude-video repository provides a video processing pipeline that balances token cost against visual richness through configurable detail modes. The frame cap logic for different detail modes is implemented in the configuration layer and consumed during frame extraction to prevent excessive token usage. This logic centers on the frame_cap() function defined in skills/watch/scripts/config.py, which returns hard limits for conservative modes and None for unlimited extraction.

How Frame Cap Limits Are Defined

The frame cap dictionary is centralized in skills/watch/scripts/config.py to ensure consistent limits across the application.

The frame_cap() Function Implementation

According to the source code in skills/watch/scripts/config.py (lines 65-74), the frame_cap() function accepts a string detail mode and returns either an integer representing the maximum frame count or None for unlimited modes:

def frame_cap(detail: str) -> int | None:
    if detail == "efficient":
        return 50
    if detail == "balanced":
        return 100
    if detail == "token-burner":
        return None
    if detail == "transcript":
        return None
    return 100

Detail Mode to Frame Cap Mapping

The function maps specific detail arguments to the following extraction budgets:

• efficient: Returns 50 frames, optimized for fast processing and minimal token consumption
• balanced: Returns 100 frames, serving as the default mid-range budget
• token-burner: Returns None, allowing unlimited frames to maximize visual detail
• transcript: Returns None, removing cap constraints because frame selection is driven entirely by transcript timestamps
• Unknown/invalid values: Falls back to 100 frames, matching the balanced default

How the Frame Cap Is Applied in Processing

The extraction budget is consumed in skills/watch/scripts/watch.py, where the cap influences which frame sampling engine runs and how many frames it may extract.

Budget Calculation and Engine Selection

As implemented in watch.py (lines 71-79 and 195-209), the script first retrieves the configured cap, then reserves slots for transcript-cue frames before applying the remaining budget to the visual extraction engine:

detail = args.detail or str(config["detail"])
configured_cap = frame_cap(detail)                 # ← cap logic

...
detail_budget = max_frames if max_frames is None else max(0, max_frames - len(cue_frames))
...
if detail != "transcript" and video_path and detail_budget != 0:
    engine_label = "keyframes" if detail == "efficient" else "scene-aware frames"
    # engine runs with max_frames=detail_budget

When detail_budget is calculated as 0, the visual extraction engine is skipped entirely. For non-zero budgets, the system selects between keyframe-based sampling (for efficient mode) and scene-aware sampling (for balanced mode), passing the remaining budget as max_frames.

Practical Usage Examples

Developers can interact with the frame cap logic programmatically or via the command-line interface.

Python API Examples

Import the frame_cap function directly from the configuration module to inspect limits before processing:

>>> from skills.watch.scripts.config import frame_cap
>>> frame_cap("efficient")
50
>>> frame_cap("balanced")
100
>>> frame_cap("token-burner")  # unlimited

>>> frame_cap("transcript")    # unlimited

>>> frame_cap("unknown")       # falls back to balanced default

100

CLI Usage Examples

Invoke the watch command with the --detail flag to specify the desired extraction budget:

# Use the “balanced” cap (default) – up to 100 frames

watch video.mp4 --detail balanced

# Request the “efficient” mode – at most 50 frames

watch video.mp4 --detail efficient

# Keep every possible frame – no cap

watch video.mp4 --detail token-burner

Summary

• The frame cap logic for different detail modes is defined in skills/watch/scripts/config.py through the frame_cap() function.
• Efficient mode limits extraction to 50 frames, while balanced mode allows 100 frames.
• Token-burner and transcript modes return None, indicating unlimited frame extraction.
• Invalid detail arguments fall back to the balanced default of 100 frames.
• The cap is consumed in skills/watch/scripts/watch.py to calculate detail_budget and select the appropriate sampling engine (keyframes vs. scene-aware).

Frequently Asked Questions

What happens if I pass an invalid detail mode to frame_cap()?

The function returns 100, which matches the balanced default. This fallback ensures the pipeline never crashes on typos and maintains reasonable token costs.

Why does transcript mode have no frame cap?

Transcript mode relies on timestamp data to drive frame selection rather than visual sampling algorithms. Since frames are extracted only at specific speech events, the limit is naturally controlled by the transcript length, making an artificial cap unnecessary.

How does the efficient mode differ from balanced mode beyond the frame cap?

In skills/watch/scripts/watch.py, efficient mode triggers the keyframes engine, which extracts only video keyframes to minimize processing overhead. Balanced mode uses scene-aware sampling, which analyzes visual content to detect changes and provides richer context at the cost of higher token usage.

Where is the frame cap actually enforced during extraction?

The cap is enforced in skills/watch/scripts/watch.py when calculating detail_budget, which is passed as max_frames to the frame extraction utilities in skills/watch/scripts/frames.py. The shared _even_sample logic in frames.py respects this budget parameter to limit the final output.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}