When to Use Efficient Mode vs Balanced Detail Mode for Frame Analysis

Use efficient mode when you need fast, low-cost keyframe extraction for quick previews, and balanced mode when you require scene-aware coverage with higher fidelity for detailed visual analysis.

The watch skill in the bradautomates/claude-video repository provides three detail levels for video frame extraction, with efficient and balanced representing the two primary options for production use. Choosing between efficient mode versus balanced detail mode for frame analysis depends entirely on your trade-off requirements between processing speed, token consumption, and visual coverage accuracy. This guide examines the implementation details in the source code to help you select the appropriate mode for your specific use case.

Core Differences Between Efficient and Balanced Modes

Extraction Strategy

The fundamental architectural difference lies in how each mode samples the video stream.

Efficient mode decodes only keyframes (I-frames) using ffmpeg -skip_frame nokey. Since keyframes are self-contained and do not require decoding previous frames, this approach is computationally cheap and extremely fast. The implementation resides in extract_keyframes() within skills/watch/scripts/frames.py.

Balanced mode performs full-decode scene-change detection using FFmpeg's scene filter. This analyzes actual visual content changes rather than compression artifacts. The function extract_scene_or_uniform() in skills/watch/scripts/frames.py first calls extract_scene_candidates() to detect cuts, and only falls back to uniform sampling if fewer than SCENE_MIN_FRAMES (8) scene changes are detected.

Performance and Token Costs

Metric	Efficient Mode	Balanced Mode
Frame Cap	50 frames (hard limit)	100 frames (hard limit)
Typical Runtime	~0.5 seconds for short clips	~20 seconds for 48-minute clips
Speed Factor	Up to 40× faster than scene modes	Baseline
Token Usage	Minimal (sparse keyframes only)	Moderate (denser scene-aware coverage)

In skills/watch/scripts/config.py, the default setting is DEFAULT_DETAIL = "balanced", reflecting the repository's recommendation for general-purpose use.

When to Use Efficient Mode

Select efficient mode in the following scenarios:

• Quick previews where you need immediate visual context without comprehensive coverage
• Short videos (< 2 minutes) or low-motion content where keyframes already capture sufficient visual information
• Tight token budgets requiring minimized image-token consumption
• Rapid iteration workflows where speed matters more than frame density

The mode enforces a fixed budget of 50 frames regardless of video length, meaning longer videos become progressively sparser. If the video contains fewer than KEYFRAME_MIN (4) keyframes, the engine automatically falls back to uniform sampling to ensure baseline coverage.

When to Use Balanced Detail Mode

Choose balanced mode when:

• Visual fidelity is critical for accurate scene cut detection, especially in longer or fast-cut videos
• The narrative depends on visual transitions (e.g., slide decks, screen recordings, action sequences)
• You require denser coverage for longer clips, as the budget scales with video duration via auto_fps() or auto_fps_focus() functions
• Token cost and processing time are acceptable trade-offs for comprehensive analysis

The 100-frame cap provides twice the coverage of efficient mode, with intelligent distribution based on actual scene changes rather than temporal compression artifacts.

Implementation Details

Key Functions in frames.py

The core logic resides in skills/watch/scripts/frames.py:

• extract_keyframes(): Handles the efficient mode path using -skip_frame nokey and optional perceptual deduplication via dedupe_perceptual
• extract_scene_or_uniform(): Implements balanced mode logic, orchestrating scene detection and fallback sampling
• extract_scene_candidates(): Performs the computational heavy lifting of scene-change detection
• auto_fps() and auto_fps_focus(): Calculate frame distribution rates for balanced mode based on video duration or user-specified ranges

Configuration and Defaults

Configuration management occurs in skills/watch/scripts/config.py, where DEFAULT_DETAIL = "balanced" establishes the conservative, high-fidelity default. The CLI entry point skills/watch/scripts/watch.py parses the --detail argument and dispatches to the appropriate extraction engine.

Code Examples

Command-Line Usage

# Fast keyframe extraction – ideal for quick previews

python skills/watch/scripts/watch.py my_video.mp4 --detail efficient

# Scene-aware extraction with higher fidelity

python skills/watch/scripts/watch.py my_video.mp4 --detail balanced

Environment Configuration

Set the default mode permanently via ~/.config/watch/.env:

WATCH_DETAIL=efficient   # or balanced

Programmatic API

from pathlib import Path
from skills.watch.scripts import frames

video = Path("presentation.mp4")
out_dir = Path("extracted_frames")

# Efficient mode: keyframes only, max 50 frames

keyframes, meta = frames.extract_keyframes(
    video_path=str(video),
    out_dir=out_dir,
    max_frames=50,
    dedup=True,
)

# Balanced mode: scene-change detection, max 100 frames

scene_frames, meta = frames.extract_scene_or_uniform(
    video_path=str(video),
    out_dir=out_dir,
    fps=2.0,
    target_frames=100,
    resolution=512,
    max_frames=100,
    dedup=True,
)

Summary

• Efficient mode extracts only keyframes via extract_keyframes(), providing 40× faster processing with a 50-frame cap, ideal for quick previews and token-sensitive workflows.
• Balanced mode uses extract_scene_or_uniform() with full scene-change detection, offering up to 100 frames of intelligent coverage for high-fidelity analysis.
• Both modes include automatic fallback to uniform sampling when content is too static (efficient falls back below 4 keyframes; balanced falls back below 8 scene changes).
• The default configuration in config.py sets balanced as the standard, reflecting its suitability for most production tasks requiring reliable visual context.

Frequently Asked Questions

What happens if a video has very few scene changes in balanced mode?

If the scene engine detects fewer than SCENE_MIN_FRAMES (8) cuts, extract_scene_or_uniform() automatically falls back to uniform sampling across the video timeline. This ensures you still receive meaningful coverage even for static content like lecture recordings or single-shot interviews.

Can I exceed the 50 or 100 frame limits in these modes?

No, the frame caps are hard-coded limits designed to control token costs and API usage. The max_frames=50 parameter in extract_keyframes() and max_frames=100 in extract_scene_or_uniform() enforce these boundaries regardless of video length or content density.

Why is balanced mode the default instead of efficient mode?

According to the repository's SKILL.md and config.py, balanced mode provides the best compromise between speed and accuracy for general-purpose video analysis. While efficient mode is significantly faster, the risk of missing critical visual transitions in longer videos outweighs the performance benefits for most use cases.

How does efficient mode handle videos with compression artifacts or few keyframes?

If the video contains fewer than KEYFRAME_MIN (4) keyframes, the extract_keyframes() function automatically triggers uniform sampling fallback. This safety mechanism ensures that highly compressed or unusual video formats still return usable frame data rather than empty results.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}