# How watch.py Orchestrates the Download, Frame Extraction, and Transcript Pipeline in claude-video

> Discover how watch.py orchestrates the claude-video pipeline. This Python script manages download, frame extraction, and transcript generation for markdown reports.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: how-to-guide
- Published: 2026-07-25

---

**[`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) serves as the main entry point for the `/watch` skill, coordinating a 13-stage pipeline that transforms video URLs or local files into markdown reports containing extracted frames and searchable transcripts.**

The [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) script in the **claude-video** repository acts as the central orchestrator for the video processing workflow. It wires together specialized helper modules to handle everything from downloading content via yt-dlp to extracting frames based on scene changes and generating transcripts. Understanding how [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) coordinates these operations reveals the architecture behind the `/watch` slash command used across Claude Code, Codex, and Cursor environments.

## CLI Argument Parsing and Configuration Loading

The pipeline begins in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) with the **`main()`** function handling CLI argument parsing via `argparse`. This captures options for source location, resolution, detail level, timestamps, time ranges, and Whisper backend selection.

Immediately after parsing, [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) loads user preferences through **`get_config`** and **`frame_cap`** from [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py). These functions resolve the `--detail` preset (efficient, balanced, or token-burner) and compute a **frame budget** (`max_frames`) that governs all subsequent extraction decisions. This budget mechanism ensures the tool respects token limits and processing constraints before any heavy operations begin.

## Source Resolution and Caption Detection

Before downloading, [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) determines whether the source is remote or local using **`is_url`** from [`skills/watch/scripts/download.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/download.py). For URL-based sources, the orchestrator attempts an optimization: **`fetch_captions`** pulls metadata and VTT subtitles via yt-dlp without downloading the actual video file.

If subtitles exist, **`parse_vtt`** from [`skills/watch/scripts/transcribe.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/transcribe.py) converts them into structured `transcript_segments`. This **caption-only check** allows the pipeline to bypass video download entirely when the user only needs transcript data, significantly reducing processing time and bandwidth.

## Video Download and Metadata Extraction

When frame extraction is required or captions are unavailable, [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) triggers **`download`** or **`download_url`** from [`skills/watch/scripts/download.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/download.py). This stage retrieves either the full video or audio-only streams depending on the detail mode requirements, returning a `video_path` and optional subtitle file.

Following successful download, the orchestrator calls **`get_metadata`** from [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py), which invokes `ffprobe` to extract technical specifications including duration, resolution, codec information, and audio track presence. This metadata feeds into the frame budgeting system and validates time-range constraints.

## Frame Budgeting and Time-Range Calculation

With video metadata available, [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) processes temporal parameters. **`parse_time`** converts human-readable `--start` and `--end` arguments into seconds, while **`auto_fps`** or **`auto_fps_focus`** from [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) calculates an appropriate frames-per-second rate that respects the `max_frames` budget across the effective duration.

This dynamic calculation ensures that whether analyzing a 30-second clip or a two-hour lecture, the frame extraction density adjusts to stay within configurable limits while maximizing coverage.

## The Frame Extraction Pipeline

The orchestration sequences frame extraction in two distinct phases:

### Cue-Frame Extraction

If the user specified `--timestamps`, [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) reserves budget for **cue frames** using **`extract_at_timestamps`** from [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py). This extracts single frames at precise transcript-cued moments (e.g., when the speaker says "look at this diagram"). These frames are reserved against the frame cap before the main extraction engine runs.

### Detail-Engine Processing

Depending on the `--detail` preset, [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) delegates to specific extraction engines in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py):

- **Efficient mode**: Uses **`extract_keyframes`** to grab only codec keyframes, minimizing processing.
- **Balanced or Token-burner modes**: Uses **`extract_scene_or_uniform`**, which attempts scene-change detection first, falling back to uniform sampling if scene detection fails or exhausts the budget.

Both engines respect the remaining **`detail_budget`** calculated earlier, ensuring the total frame count stays within limits.

### Frame Merging and Deduplication

After extraction, **`merge_frames`** from [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) combines cue frames with detail-engine frames, preserving chronological order and removing temporal duplicates. Unless `--no-dedup` is specified, [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) runs deduplication logic to eliminate visually similar frames, which is particularly useful for slide-heavy presentations where minimal visual change occurs between frames.

## Transcript Generation and Whisper Fallback

For transcripts, [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) implements a cascading strategy. If `fetch_captions` succeeded earlier, those parsed segments flow directly to the report. Otherwise, the orchestrator checks if Whisper is disabled; if not, it proceeds to audio transcription.

The orchestrator calls **`load_api_key`** and **`transcribe_video`** from [`skills/watch/scripts/whisper.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py). These functions select between Groq and OpenAI backends based on availability and user preference, then execute the Whisper API against the downloaded audio. This fallback mechanism ensures transcript availability even for videos without native captions.

## Report Generation and Cleanup

In the final stage, [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) assembles the markdown report using **`format_transcript`** and **`format_time`** helpers. The report includes source information, duration, resolution, frame statistics with extraction reasons (scene-change, uniform, transcript-cue), and the formatted transcript.

All temporary files reside in a working directory generated via `tempfile.mkdtemp`, which [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) cleans up after report generation, leaving only the final markdown output containing embedded frame references and transcript data.

## Practical Usage Examples

```bash

# Basic usage – download a YouTube video, extract balanced frames, and show captions

watch https://www.youtube.com/watch?v=xyz123

# Focus on a 30-second segment with higher resolution and custom fps

watch https://youtu.be/xyz123 --start 01:23 --end 01:53 \
      --resolution 1024 --fps 1.5 --detail balanced

# Extract precise frames at transcript-cued timestamps (e.g., "look here" moments)

watch https://youtu.be/xyz123 --timestamps 00:45,02:10,03:05

# Force Whisper fallback (Groq or OpenAI) when no subtitles are available

watch ./local_video.mp4 --whisper groq

# Disable near-duplicate removal (useful for slide decks)

watch ./presentation.mp4 --no-dedup

```

## Key Architectural Components

| File | Role |
|------|------|
| [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) | Main orchestration script parsing CLI arguments and coordinating the full pipeline |
| [`skills/watch/scripts/download.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/download.py) | URL detection, caption fetching, and video/audio download via yt-dlp |
| [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py) | Metadata probing, fps calculation, frame extraction, and deduplication logic |
| [`skills/watch/scripts/transcribe.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/transcribe.py) | VTT subtitle parsing and transcript formatting |
| [`skills/watch/scripts/whisper.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py) | API key management and Whisper backend selection |
| [`skills/watch/scripts/config.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/config.py) | User-configurable defaults and detail presets |

## Summary

- **watch.py** acts as the linear orchestrator for the entire `/watch` skill, managing a 13-stage pipeline from argument parsing to final markdown output.
- The script prioritizes efficiency by attempting caption-only fetches before downloading full video files, falling back to yt-dlp downloads only when necessary.
- Frame extraction follows a strict budget system using `max_frames` and dynamic fps calculation, with separate handling for cue frames and detail-engine frames.
- All heavy lifting is delegated to specialized modules: [`download.py`](https://github.com/bradautomates/claude-video/blob/main/download.py) for acquisition, [`frames.py`](https://github.com/bradautomates/claude-video/blob/main/frames.py) for visual processing, and [`whisper.py`](https://github.com/bradautomates/claude-video/blob/main/whisper.py) for audio transcription.
- The architecture remains host-agnostic, functioning uniformly across Claude Code, Codex, Cursor, and other environments without host-specific dependencies.

## Frequently Asked Questions

### How does watch.py decide whether to download the full video or just captions?

[`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) calls `is_url` to identify remote sources, then immediately attempts `fetch_captions` from [`skills/watch/scripts/download.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/download.py). If the video source provides VTT subtitles and the user only requested transcript data (or the detail mode allows it), the pipeline skips the download phase entirely. Full video download occurs only when frame extraction is required or when no captions exist and Whisper transcription is needed.

### What happens if I specify timestamps that exceed the frame budget?

[`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) reserves the cue-frame budget first using `extract_at_timestamps` before running the detail engine. If timestamp requests exceed `max_frames`, the system prioritizes the explicitly requested cue frames and reduces sampling in the detail-engine phase. The `auto_fps` and `auto_fps_focus` functions recalculate sampling rates to accommodate both cue frames and scene-extraction within the remaining budget.

### Can watch.py process local video files, or only URLs?

Yes, [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) handles both sources. The `is_url` function from [`skills/watch/scripts/download.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/download.py) returns false for local filesystem paths, causing the orchestrator to skip the download phase and proceed directly to metadata extraction via `get_metadata` in [`skills/watch/scripts/frames.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/frames.py). Local processing supports all the same frame extraction and transcription features as URL-based workflows.

### How does the deduplication logic work, and when should I disable it?

By default, [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) runs deduplication after `merge_frames` to remove visually similar frames that fall within short temporal windows. This prevents waste when extracting from slide presentations or static scenes. Use the `--no-dedup` flag when you need to preserve every extracted frame, such as when analyzing subtle animation changes or when precise frame counts matter more than token efficiency.