Claude-Video Transcript Generation Sources: YouTube Captions vs. Whisper API

Claude-Video generates transcripts by first attempting to download existing WebVTT captions from YouTube via yt-dlp, then falling back to audio transcription using the Groq or OpenAI Whisper API when subtitles are unavailable.

Claude-Video is an open-source video processing tool that extracts spoken content for AI analysis. Understanding its dual-source approach to transcript generation reveals how the system balances speed and accuracy when handling video URLs. The implementation prioritizes existing subtitle files before incurring API costs for audio transcription.

Primary Source: YouTube WebVTT Captions

When processing a video URL, Claude-Video first attempts to retrieve existing subtitle files. This approach avoids unnecessary API calls and preserves the original timing accuracy provided by content creators.

Downloading and Parsing Subtitles

The fetch_captions function in skills/watch/scripts/watch.py (lines 98-105) uses yt-dlp to download available WebVTT files from YouTube or other supported platforms. If successful, the tool passes the subtitle_path to parse_vtt in skills/watch/scripts/transcribe.py (lines 24-52).

The parser processes the WebVTT file to remove duplicate cues and normalize timing data. It returns a standardized list of segment objects containing start, end, and text fields. This normalization ensures downstream components receive consistent data regardless of the original caption format.

# From transcribe.py - parse_vtt normalizes caption data

segments = parse_vtt(subtitle_path)

# Returns: [{"start": 0.0, "end": 5.2, "text": "Hello world"}, ...]

Fallback Source: Whisper API Transcription

When no captions exist or parsing fails, Claude-Video extracts audio and sends it to a Whisper speech-to-text service. This fallback ensures transcript generation works for any video with an audio track.

Audio Extraction and API Selection

The transcribe_video function in skills/watch/scripts/whisper.py handles audio processing. It extracts a mono 16kHz MP3 from the video source, optimizing for transcription accuracy while minimizing file size. The load_api_key function (lines 14-30) selects the first available API key, preferring Groq over OpenAI when both are configured.

# API key selection prioritizes Groq

api_key = load_api_key()  # Checks GROQ_API_KEY first, then OPENAI_API_KEY

Chunking and Response Normalization

For long videos, the system automatically splits audio into manageable chunks before transmission. After receiving the JSON response from the Whisper endpoint, _segments_from_response converts the API-specific format into the same {start, end, text} structure used by the caption parser. This normalization occurs in skills/watch/scripts/whisper.py (lines 65-78).

Implementation Workflow in watch.py

The orchestration logic in skills/watch/scripts/watch.py implements a clear priority system. Lines 39-52 contain the decision block that evaluates caption availability before triggering the Whisper fallback.

The workflow follows this sequence:

1. Attempt caption retrieval – Call fetch_captions and validate the returned subtitle_path
2. Parse existing subtitles – If available, process through parse_vtt to generate segments
3. Trigger audio transcription – If captions fail, verify audio exists and call transcribe_video
4. Apply time filtering – Optionally pass results through filter_range in transcribe.py to isolate specific segments
5. Format output – Render the final transcript using format_transcript for markdown compatibility

Configuration and Usage Examples

Claude-Video provides command-line options to control transcript generation behavior. By default, the tool attempts to use existing captions first.

# Default: Use YouTube captions if available

python -m skills/watch/scripts/watch.py "https://www.youtube.com/watch?v=abc123"

To bypass captions and force Whisper transcription, specify the provider explicitly:

# Force Whisper using OpenAI (requires OPENAI_API_KEY)

python -m skills/watch/scripts/watch.py "https://www.youtube.com/watch?v=abc123" \
    --whisper openai

Both commands produce a markdown report containing a Transcript section. The system handles source-specific quirks internally, presenting unified output regardless of whether the content originated from WebVTT files or API transcription.

Summary

• Claude-Video uses a two-tier priority system for transcript generation: existing captions first, Whisper API second.
• WebVTT parsing occurs in transcribe.py via parse_vtt, which deduplicates cues and normalizes timing data.
• Whisper transcription in whisper.py handles audio extraction, Groq/OpenAI API selection, and response normalization through _segments_from_response.
• The decision logic resides in watch.py, which orchestrates the workflow and ensures consistent segment formatting across both sources.
• Users can override defaults via command-line flags to force specific transcription methods.

Frequently Asked Questions

What happens if a YouTube video has no captions available?

If fetch_captions returns no subtitle file, Claude-Video automatically proceeds to the Whisper fallback. The system extracts the audio track and sends it to the configured Whisper API provider (Groq preferred over OpenAI) to generate transcripts from scratch.

Which Whisper API provider does Claude-Video prefer?

According to the source code in whisper.py, the load_api_key function checks for GROQ_API_KEY before OPENAI_API_KEY. This prioritization allows users to leverage Groq's typically faster inference speeds and competitive pricing while maintaining OpenAI as a backup option.

How does Claude-Video handle long videos with Whisper?

The transcribe_video function implements audio chunking logic to split long videos into segments that comply with API size limits. It extracts mono 16kHz MP3 audio and determines whether to upload the file whole or split it into chunks based on duration and size constraints.

Can I force Whisper transcription even if captions exist?

Yes. By passing the --whisper flag with either groq or openai as the value, you bypass the caption retrieval logic entirely. This forces the system to extract audio and use the specified API provider regardless of whether WebVTT subtitles are available for the video.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}