Can Claude Video Transcribe Videos Without Existing Captions? A Complete Guide

Yes, Claude Video can transcribe videos without existing captions by automatically falling back to OpenAI or Groq Whisper transcription when no embedded or fetched subtitles are available.

Claude Video is an open-source tool that enables AI-powered video analysis through the Claude API. When processing videos that lack existing caption files, the system seamlessly transitions to on-demand audio transcription, ensuring you can extract text content from any video source.

How the Transcription Fallback Works

The /watch skill in skills/watch/scripts/watch.py implements a robust two-stage pipeline that prioritizes existing captions before invoking Whisper.

Step 1: Checking for Existing Subtitles

First, download.fetch_captions() attempts to retrieve any available subtitle files from the video source. If the function discovers a VTT file, it parses the content using parse_vtt and populates the transcript segments. If no subtitle file is found, the subtitle_path entry remains None and the transcript stays empty.

Step 2: Triggering the Whisper Fallback

After the initial fetch, watch.py evaluates whether transcript_segments is empty. If no captions were discovered and the user has not disabled transcription via the --no-whisper flag, the script proceeds to the Whisper fallback mechanism. This detection logic appears at lines 39-57 of skills/watch/scripts/watch.py.

Step 3: Audio Extraction and Transcription

The system prepares the audio through several sub-steps managed by skills/watch/scripts/whisper.py:

1. API Key Resolution: whisper.load_api_key() checks for a GROQ_API_KEY environment variable first, falling back to OPENAI_API_KEY. If neither is found, the user is prompted to run the setup helper (lines 65-71).

2. Audio Processing: whisper.extract_audio() invokes ffmpeg to generate a mono 16 kHz MP3 at approximately 64 kbps. This format complies with Whisper's upload constraints while minimizing file size (lines 15-31).

3. Chunked Upload: whisper.transcribe_video() handles the API communication. For files exceeding 24 MiB, the audio is automatically chunked. The function posts the data to the selected endpoint (Groq or OpenAI), receives a verbose JSON response, and converts the results into the internal {start, end, text} segment format (lines 37-45, 51-66).

Step 4: Integrating the Transcript

The returned segments are filtered to match any user-specified time range using --start and --end parameters. The final report explicitly labels the transcript source as "whisper (groq)" or "whisper (openai)" to indicate the transcription provider used.

Prerequisites for Transcribing Videos Without Captions

To successfully transcribe videos that lack existing captions, ensure your environment meets these requirements:

• ffmpeg and ffprobe must be installed and available in your system PATH for audio extraction.
• API Key Configuration: Either GROQ_API_KEY or OPENAI_API_KEY must be set in your environment or stored in ~/.config/watch/.env.
• Enable Whisper: Do not pass the --no-whisper flag, which disables the fallback and forces frames-only mode.

If these conditions are unmet, Claude Video reports that no transcript is available and continues with visual frame analysis only.

Usage Examples

Basic Usage (Automatic Fallback)

Run the watch command on any URL. The tool attempts caption discovery first, then automatically extracts audio and calls Whisper if needed:

watch https://www.youtube.com/watch?v=example_video

Force a Specific Backend

Override the default Groq preference and use OpenAI's Whisper API explicitly:

watch https://example.com/video.mp4 --whisper openai

Disable Transcription (Frames-Only Mode)

Skip the Whisper fallback entirely when you lack API credentials or only need visual analysis:

watch https://example.com/video.mp4 --no-whisper

Transcribe a Specific Time Window

Limit transcription to a specific segment while still using Whisper if no captions exist:

watch https://example.com/video.mp4 --start 1:30 --end 2:15

Summary

• Claude Video transcribes captionless videos by falling back to Whisper via Groq or OpenAI when download.fetch_captions() returns no results.
• The fallback triggers automatically in watch.py when transcript_segments is empty and --no-whisper is not set.
• Audio extraction uses ffmpeg to create Whisper-compatible 16 kHz mono MP3 files.
• Chunking support allows transcription of videos larger than 24 MiB by splitting audio automatically.
• Source attribution clearly labels transcripts as "whisper (groq)" or "whisper (openai)" in the final output.

Frequently Asked Questions

What happens if I don't have an API key configured?

If neither GROQ_API_KEY nor OPENAI_API_KEY is found, whisper.load_api_key() prompts you to run the setup helper. Without valid credentials, Claude Video skips transcription and operates in frames-only mode.

Can I choose between Groq and OpenAI for transcription?

Yes. While the system prefers Groq by default, you can force a specific backend using the --whisper flag followed by either groq or openai to route requests to your preferred provider.

Does Claude Video download the entire video to transcribe it?

Yes. The tool uses yt-dlp via download.py to retrieve the video file, then ffmpeg extracts the audio track. For large files, the audio is chunked during the upload phase to stay within API limits.

What audio format does Claude Video use for Whisper transcription?

The system converts video audio to a mono 16 kHz MP3 at approximately 64 kbps using whisper.extract_audio(). This format balances quality with file size to ensure compatibility with Whisper's upload requirements while minimizing API costs.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}