How the Transcript-First Approach Works in Claude Video

The transcript-first approach in Claude Video prioritizes textual transcripts over video downloads, fetching captions via yt-dlp first and only downloading video data when subtitles are unavailable or when the user explicitly requests visual frames.

Claude Video’s /watch skill implements a transcript-first workflow designed to minimize bandwidth usage and accelerate response times. According to the bradautomates/claude-video source code, this architecture attempts to extract text transcripts before processing any video frames, ensuring lightweight operation when only textual content is needed.

Core Mechanism of the Transcript-First Workflow

The transcript-first strategy operates through a prioritized pipeline defined in skills/watch/scripts/watch.py. This design ensures that captions are always attempted before any costly video processing begins.

Fetching Existing Captions

When a user provides a URL, the script immediately invokes fetch_captions (lines 97-104) to retrieve available subtitle files via yt-dlp. If a .vtt file is found, the parse_vtt function processes the raw WebVTT data, and format_transcript converts it into a clean, timestamped transcript (lines 102-105). The system records the transcript source as captions, indicating that no audio extraction was necessary.

Bypassing Video Downloads

If the user specifies --detail transcript and a valid transcript exists from the caption fetch, the script sets audio_only = False and video_path = None (lines 110-113). This conditional logic prevents unnecessary video downloads entirely, saving bandwidth and processing time when only text is required.

Whisper Fallback Pipeline

When captions are missing or fail to parse, the system falls back to the transcribe_video helper (lines 39-49). This fallback executes only after an audio-only download (or full video download when --detail transcript is not used), ensuring that Whisper transcription occurs only when necessary. The transcript source is then recorded as whisper in the final report.

On-Demand Visual Cue Extraction

The transcript-first approach does not preclude visual analysis. When users supply --timestamps, the script extracts frames at specific timestamps after the transcript is available. These transcript-cue frames are extracted via the logic in lines 76-84 and count against the frame budget defined in skills/watch/scripts/config.py. This maintains the lightweight philosophy while allowing visual verification of specific moments mentioned in the transcript.

Reporting and Source Attribution

At completion, the markdown report lists the transcript segment count and indicates the generation source—either captions or whisper (lines 310-315). If transcription fails entirely, the script outputs a diagnostic hint directing users to run Whisper setup (lines 60-65), ensuring transparent feedback about data sources.

Practical Implementation Examples

The following commands demonstrate the transcript-first workflow in practice:

# Transcript-only mode using existing captions (no video download)

watch https://youtu.be/abc123 --detail transcript

# Force Whisper fallback when captions are unavailable

watch https://example.com/video.mp4 --detail transcript

# Extract cue frames at specific timestamps after transcript generation

watch https://youtu.be/abc123 --detail transcript --timestamps 0:45,1:30

The Python implementation mirrors this logic:

from watch import fetch_captions, parse_vtt, format_transcript, transcribe_video

# Priority 1: Try existing captions

dl = fetch_captions(url, download_dir)
if dl.get("subtitle_path"):
    segs = parse_vtt(dl["subtitle_path"])
    transcript = format_transcript(segs)
else:
    # Priority 2: Whisper fallback

    segs, _ = transcribe_video(video_path, audio_path, backend="groq", api_key="...")
    transcript = format_transcript(segs)

Summary

• The transcript-first approach attempts to fetch existing captions via fetch_captions before any video download occurs.
• When --detail transcript is specified and captions exist, the script bypasses video entirely by setting video_path = None.
• Whisper transcription serves as a fallback only when captions are missing, triggered via transcribe_video.
• Visual frames are extracted on-demand using --timestamps, creating transcript-cue frames that respect the frame budget.
• Final reports explicitly identify the transcript source as either captions or whisper for complete transparency.

Frequently Asked Questions

What happens if a video has no captions?

If yt-dlp returns no subtitle files, the script automatically proceeds to the transcribe_video fallback. This downloads the audio track and processes it through the Whisper API (via Groq or local backend) to generate a transcript before any frame extraction occurs.

Can I use the transcript-first approach without installing Whisper?

Yes. If the target video has existing captions (like most YouTube videos), the --detail transcript flag will retrieve and format those captions without ever invoking Whisper or downloading video data. Whisper is only required as a fallback for uncaptioned content.

How does the frame budget interact with transcript-cue frames?

Transcript-cue frames extracted via --timestamps are counted against the total frame budget defined in skills/watch/scripts/config.py. This ensures that even when operating in transcript-first mode, visual requests remain constrained by the configured frame_cap limits.

Where is the transcript source recorded in the output?

The final markdown report generated in watch.py (lines 310-315) explicitly lists the transcript source as either captions or whisper, along with the total segment count. This metadata appears in the skill output to confirm which extraction path was utilized.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}