# How Claude-video's /watch Endpoint Handles transcript, efficient, balanced, and token-burner Detail Modes

> Explore how Claude-video's /watch endpoint manages transcript, efficient, balanced, and token-burner detail modes for optimized video processing. Learn about frame extraction budgets and token usage.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-07-10

---

**Claude-video's /watch endpoint routes video processing through four distinct detail modes—transcript, efficient, balanced, and token-burner—that determine frame extraction budgets, algorithm selection, and token usage based on a hierarchical configuration system.**

Claude-video is an open-source video analysis tool that processes video content through a flexible detail mode system. The `/watch` endpoint in the bradautomates/claude-video repository allows users to control frame extraction and transcription behavior through four distinct modes. Understanding how these modes resolve and their specific frame budgets helps optimize token usage and processing time for different video analysis tasks.

## Detail Mode Configuration Hierarchy

The detail mode follows a strict resolution order defined in [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py)【/cache/repos/github.com/bradautomates/claude-video/main/skills/watch/scripts/config.py#L12-L60】. The system evaluates potential sources in priority order, falling back to subsequent options only when higher-priority values are absent.

### Resolution Order

The mode selection follows this hierarchy:

1. **CLI flag** `--detail` (highest priority)
2. **Environment variable** `WATCH_DETAIL` (or values in `~/.config/watch/.env`)
3. **Default value** `"balanced"` (fallback)

The `get_config()` function implements this logic:

```python
DEFAULT_DETAIL = "balanced"
DETAILS = {"transcript", "efficient", "balanced", "token-burner"}

def get_config() -> dict:
    detail = (
        os.environ.get("WATCH_DETAIL")
        or file_values.get("WATCH_DETAIL")
        or DEFAULT_DETAIL
    )
    if detail not in DETAILS:
        detail = DEFAULT_DETAIL
    return {"detail": detail, "config_file": str(CONFIG_FILE)}

```

### Configuration Validation

If the resolved value does not exist in the `DETAILS` set, the system **safely defaults to "balanced"**. This validation prevents invalid modes from crashing the pipeline while ensuring predictable behavior.

## Frame Budget and Extraction Algorithms

The `frame_cap()` function in [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py)【/cache/repos/github.com/bradautomates/claude-video/main/skills/watch/scripts/config.py#L65-L74】 maps each mode to a numeric cap or `None` for uncapped extraction:

- **transcript**: `None` (no frames unless timestamps provided)
- **efficient**: `50` (fast keyframe extraction)
- **balanced**: `100` (scene-aware extraction)
- **token-burner**: `None` (uncapped scene-aware extraction)

### Transcript Mode

When `detail="transcript"`, the pipeline **skips frame extraction entirely** unless the user supplies explicit timestamps via `--timestamps`. The system sets `audio_only = True` to avoid downloading video streams unnecessarily. This mode prioritizes minimal token usage by processing only the spoken content through either existing subtitles or Whisper API transcription.

### Efficient Mode

The **efficient** mode caps frame extraction at **50 frames** and invokes the `extract_keyframes()` function. This algorithm uses fast keyframe detection to provide quick visual overviews with minimal processing time. This mode suits rapid content scanning where scene-level detail is unnecessary.

### Balanced Mode

As the default configuration, **balanced** mode allocates a **100-frame budget** and utilizes `extract_scene_or_uniform()`. This scene-aware algorithm attempts to capture representative frames across video segments while respecting the hard cap. The system intelligently distributes frames across the timeline to maximize coverage within the 100-frame limit.

### Token-burner Mode

The **token-burner** mode removes the frame cap entirely (`None`), allowing `extract_scene_or_uniform()` to generate as many frames as the scene-change detector identifies. This can produce **hundreds of frames** for long videos, significantly increasing token consumption. The system emits a warning when processing generates more than 250 frames【/cache/repos/github.com/bradautomates/claude-video/main/skills/watch/scripts/watch.py#L19-L24】 to alert users about potential cost implications.

## Execution Flow in watch.py

The main orchestration logic in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py)【/cache/repos/github.com/bradautomates/claude-video/main/skills/watch/scripts/watch.py#L71-L73】 implements the mode-specific processing pipeline.

### Frame Budget Calculation

After resolving the detail mode, the script calculates the effective `detail_budget` by accounting for any cue frames requested via `--timestamps`【/cache/repos/github.com/bradautomates/claude-video/main/skills/watch/scripts/watch.py#L95-L99】:

```python

# Subtract cue frames from the budget

if detail == "transcript":
    detail_budget = 0  # Unless timestamps provided

else:
    detail_budget = frame_cap(detail)

```

### Engine Selection Logic

The script selects extraction engines based on the resolved mode【/cache/repos/github.com/bradautomates/claude-video/main/skills/watch/scripts/watch.py#L95-L105】【/cache/repos/github.com/bradautomates/claude-video/main/skills/watch/scripts/watch.py#L190-L206】:

- **`detail == "efficient"`** → Calls `extract_keyframes()` for fast keyframe extraction
- **`detail in {"balanced", "token-burner"}`** → Calls `extract_scene_or_uniform()` for scene-aware processing
- **`detail == "transcript"`** → Skips frame extraction unless cue timestamps exist

When timestamps are provided alongside transcript mode, those specific frames are extracted as **cue frames** and appear in the final report despite the transcript setting【/cache/repos/github.com/bradautomates/claude-video/main/skills/watch/scripts/watch.py#L76-L84】.

## Transcript Processing Pipeline

The transcript handling operates independently of frame extraction, utilizing two data sources:

1. **Existing subtitles**: If `dl["subtitle_path"]` exists, the system parses VTT files using `parse_vtt()`
2. **Whisper API**: When no subtitles exist and `--no-whisper` is not set, the script calls `transcribe_video()` to generate transcripts

In transcript-only mode, the final report displays **"Frames: skipped (transcript detail)"** while showing the transcript segment count and source (captions or Whisper)【/cache/repos/github.com/bradautomates/claude-video/main/skills/watch/scripts/watch.py#L88-L99】.

## Practical Usage Examples

The following commands demonstrate each detail mode:

```bash

# Fast keyframe extraction (max 50 frames)

watch https://youtu.be/xyz --detail efficient

# Scene-aware extraction with 100-frame cap

watch https://youtu.be/xyz --detail balanced

# Uncapped scene extraction (high token usage)

watch https://youtu.be/xyz --detail token-burner

# Transcript only (no frames)

watch https://youtu.be/xyz --detail transcript

# Transcript with specific cue frames

watch https://youtu.be/xyz --detail transcript --timestamps "0:30,1:45"

```

## Summary

- **Configuration hierarchy**: CLI flag `--detail` takes precedence over `WATCH_DETAIL` environment variable, with "balanced" as the ultimate fallback.
- **Frame budgets**: transcript (0), efficient (50), balanced (100), token-burner (uncapped).
- **Algorithm selection**: efficient uses `extract_keyframes()`; balanced and token-burner use `extract_scene_or_uniform()`; transcript skips frames unless timestamps are provided.
- **Token warnings**: The system warns when token-burner mode generates more than 250 frames or when capped modes produce sparse coverage on long videos.
- **Source files**: Mode resolution occurs in [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py) while execution logic resides in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py).

## Frequently Asked Questions

### How does Claude-video choose which detail mode to use when I don't specify one?

The system follows a strict hierarchy: it first checks for the `--detail` CLI flag, then falls back to the `WATCH_DETAIL` environment variable (or the value in `~/.config/watch/.env`), and finally defaults to **"balanced"** if neither is set. This ensures consistent behavior across different execution environments while allowing easy overrides.

### What's the difference between efficient and balanced modes in terms of processing?

**Efficient** mode extracts up to 50 frames using fast keyframe detection, prioritizing speed over scene coverage. **Balanced** mode uses scene-aware extraction (`extract_scene_or_uniform()`) with a 100-frame cap, attempting to distribute frames evenly across the video timeline. The balanced algorithm provides better visual representation but requires more processing time to analyze scene changes.

### Can I extract specific frames while still using transcript mode?

Yes. When you provide `--timestamps` alongside `--detail transcript`, the system extracts those specific timestamps as **cue frames** while still skipping the general frame extraction pipeline. This allows you to reference specific visual moments in a video while primarily consuming the transcript content, optimizing for both precision and token efficiency.

### Why does token-burner mode warn about frame counts?

The **token-burner** mode removes the frame cap entirely, allowing the scene-detection algorithm to generate as many frames as it finds relevant. For long videos with frequent scene changes, this can produce **hundreds of frames**, significantly increasing API token consumption and costs. The warning at 250 frames serves as a safeguard to alert users before they incur unexpected expenses from uncapped frame extraction.