When Can Efficient Mode Return More Frames Than Balanced Mode in Claude-Video?

Efficient mode can return more frames than balanced mode in low-motion video when encoder-generated keyframes outnumber visually detected scene cuts, causing the keyframe-only extraction to exceed the scene-aware sampling count despite its lower 50-frame cap versus 100.

The bradautomates/claude-video repository provides a Python-based video analysis tool that offers three distinct extraction strategies controlled by the --detail flag. Understanding when efficient mode produces higher frame counts than balanced mode requires examining how each strategy interacts with video encoding characteristics, particularly in footage with minimal visual motion.

How Frame Extraction Works in Claude-Video

Claude-Video implements three detail tiers, each with distinct extraction mechanics and frame limits defined in skills/watch/scripts/config.py.

Efficient Mode: Keyframe-Only Extraction

Efficient mode leverages ffmpeg with the -skip_frame nokey parameter to extract only keyframes—frames that the video encoder marks as reference points for decoding. According to the source code in skills/watch/scripts/watch.py at line 198, this mode selects the "keyframes" engine:

engine_label = "keyframes" if detail == "efficient" else "scene-aware frames"

The frame_cap function in skills/watch/scripts/config.py (lines 65-70) restricts efficient mode to a maximum of 50 frames:

def frame_cap(detail: str) -> int | None:
    if detail == "efficient":
        return 50          # ← cap for efficient mode

    if detail == "balanced":
        return 100         # ← cap for balanced mode

Balanced Mode: Scene-Aware Sampling

Balanced mode performs a computationally heavier scene-aware pass. It decodes every frame, detects visual scene cuts using content analysis, and falls back to duration-aware uniform sampling only when cuts are sparse. This mode captures semantic transitions rather than encoding artifacts, with a higher frame cap of 100.

Why Efficient Mode Can Exceed Balanced Mode in Low-Motion Video

In low-motion video—such as screen recordings, static camera shots, or slideshows—visual content changes infrequently. However, video encoders insert periodic keyframes (typically every 2 seconds) regardless of motion to maintain decoding resilience.

This creates a counterintuitive outcome:

• Balanced mode detects few scene cuts in low-motion footage, often extracting far fewer than its 100-frame cap
• Efficient mode captures all encoder-generated keyframes, potentially hitting its 50-frame cap and occasionally exceeding the actual frame count returned by balanced mode

As documented in README.md (lines 87-93): "efficient is the speed tier… It can also return more frames than balanced on low‑motion footage (keyframes outnumber scene cuts); 'efficient' means fast extraction, not fewer frames."

Practical Example

Consider a 60-second screen recording with keyframes every 2 seconds but only 3 actual scene changes:

# Efficient mode extracts all 30 keyframes

watch.py --detail efficient screen_recording.mp4

# Output: 30 frames extracted

# Balanced mode extracts only the 3 scene cuts

watch.py --detail balanced screen_recording.mp4

# Output: 3 frames extracted

In this scenario, efficient mode returns 10 times more frames than balanced mode despite having the lower theoretical maximum.

Code Implementation Details

The extraction pipeline behavior is controlled by two critical components in the codebase.

Frame Cap Configuration

The frame_cap function in skills/watch/scripts/config.py establishes the hard limits:

def frame_cap(detail: str) -> int | None:
    if detail == "efficient":
        return 50
    if detail == "balanced":
        return 100
    if detail == "token-burner":
        return None  # Uncapped

    return 50  # Default fallback

Engine Selection Logic

In skills/watch/scripts/watch.py, the detail parameter determines which extraction engine initializes:

if detail == "efficient":
    extractor = KeyframeExtractor(skip_frame_nokey=True)
    max_frames = frame_cap("efficient")
else:
    extractor = SceneAwareExtractor()
    max_frames = frame_cap("balanced")

The KeyframeExtractor uses ffmpeg's -skip_frame nokey flag to bypass non-keyframe decoding entirely, while SceneAwareExtractor processes the full frame sequence to detect content boundaries.

Summary

• Efficient mode captures keyframes (encoder reference frames) with a 50-frame cap, using ffmpeg -skip_frame nokey
• Balanced mode detects scene cuts visually with a 100-frame cap, falling back to uniform sampling when cuts are sparse
• In low-motion video, periodic encoder keyframes often outnumber actual scene changes, causing efficient mode to return more frames than balanced mode
• The frame_cap function in skills/watch/scripts/config.py defines these limits, while skills/watch/scripts/watch.py orchestrates the extraction strategy

Frequently Asked Questions

What is the frame cap for efficient mode versus balanced mode?

Efficient mode has a 50-frame cap, while balanced mode allows up to 100 frames. This is defined in the frame_cap function in skills/watch/scripts/config.py (lines 65-70). Token-burner mode operates with no cap.

Does efficient mode always return fewer frames than balanced mode?

No. While efficient mode has a lower maximum limit, it can return more frames than balanced mode in low-motion video where encoder-generated keyframes outnumber visual scene cuts. The README.md explicitly notes that "efficient" refers to extraction speed, not necessarily frame quantity.

How does ffmpeg's skip_frame nokey parameter work in claude-video?

The -skip_frame nokey parameter instructs ffmpeg to decode only keyframes (I-frames), skipping predictive frames (P-frames) and bidirectional frames (B-frames). This makes extraction extremely fast but captures every encoder-designated reference frame, including periodic keyframes that may not represent visual scene changes.

When should I use balanced mode instead of efficient mode?

Use balanced mode when you need semantic scene detection rather than encoding artifacts, particularly for high-motion video with rapid cuts where scene-aware sampling provides better content representation. Use efficient mode for speed-critical applications or when analyzing footage where you suspect keyframe density exceeds scene change density.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}