How Claude-Video Handles Videos Without Captions or Audio Streams

Claude-Video processes videos lacking captions or transcripts through a three-tier fallback system: embedded subtitle extraction, Whisper speech-to-text generation, and graceful degradation to frame-only analysis when neither is possible.

When working with video content for AI analysis, missing transcripts create a common challenge. The bradautomates/claude-video repository implements a robust, layered approach to ensure users always receive meaningful output—even when source videos lack captions or audio entirely. This article breaks down exactly how the tool handles these edge cases based on its source code implementation.

The Transcript Acquisition Pipeline

The core orchestration happens in skills/watch/scripts/watch.py. When you run the watch command, the script executes a predictable sequence to obtain usable text from any video source.

Step 1: Embedded Caption Retrieval

For URL-based sources (primarily YouTube), Claude-Video first attempts to download existing subtitles without fetching the full video.

In watch.py lines 97-105, the script calls fetch_captions from download.py to retrieve VTT subtitle files via yt-dlp:

# From watch.py - caption fetching logic

success, caption_path = fetch_captions(video_source)
if success and caption_path:
    transcript = parse_vtt(caption_path)

When successful, parse_vtt (lines 101-104) converts the subtitle segments into a structured transcript that feeds directly into the final report. This is the fastest path—no video download, no API calls, no speech-to-text processing.

Step 2: Whisper Speech-to-Text Fallback

When embedded captions are unavailable and the user hasn't disabled Whisper via --no-whisper, Claude-Video checks for a usable audio stream.

At lines 39-52 in watch.py, the validation logic looks like this:

if not args.no_whisper and meta.get("has_audio"):
    api_key = load_api_key()
    if api_key:
        transcript = transcribe_video(video_source, api_key)

The meta.get("has_audio") check prevents wasted processing on silent videos. If audio exists, load_api_key retrieves the appropriate backend credentials (Groq or OpenAI), then transcribe_video generates the transcript on-the-fly.

The actual speech-to-text implementation lives in skills/watch/scripts/whisper.py, which abstracts the API-specific details and handles both Groq and OpenAI backends through a unified interface.

Graceful Degradation Paths

Claude-Video doesn't fail when transcript acquisition is impossible. Instead, it branches into two clear degradation modes based on why transcription failed.

Missing Whisper Configuration

If no API key is configured or the user explicitly passed --no-whisper, the script outputs a helpful diagnostic at lines 55-63:

print("No captions found. Whisper is disabled or not configured.")
print("Run: python -m skills.watch.scripts.setup")
print("to configure Whisper speech-to-text.")

This directs users to setup.py, which collects and stores API credentials for future runs.

Absent Audio Streams

When meta.get("has_audio") returns False, the code at lines 65-66 takes the most minimal path:

else:
    # No audio stream available

    transcript = None

Claude-Video skips transcription entirely and proceeds with frame extraction only. This handles screen recordings, GIF-converted videos, and other silent media without error states.

User-Facing Transcript Reporting

The final markdown report communicates transcript status unambiguously. In watch.py lines 77-84, three distinct messages cover all outcomes:

Condition	Report Message
Successful transcription (captions or Whisper)	Actual transcript content
No transcript possible, frames available	"No transcript available – proceed with frames only…"
Transcript-only mode requested, none available	"No transcript available at transcript detail…"

This transparent communication ensures users understand exactly what analysis they're receiving.

Practical Command Examples

Control the fallback behavior through CLI flags:

# Default: captions first, then Whisper if needed

watch https://www.youtube.com/watch?v=example

# Skip Whisper entirely – useful for known-silent videos or unconfigured environments

watch https://www.youtube.com/watch?v=example --no-whisper

# Transcript-only analysis – fails explicitly if no text source exists

watch https://www.youtube.com/watch?v=example --detail transcript

Key Implementation Files

Understanding the full architecture requires familiarity with these modules:

• skills/watch/scripts/watch.py — Orchestrates the entire workflow: caption fetching, Whisper invocation, and report generation
• skills/watch/scripts/download.py — Handles yt-dlp interactions; includes fetch_captions for subtitle-only retrieval
• skills/watch/scripts/whisper.py — Loads appropriate backends (Groq/OpenAI) and executes transcribe_video
• skills/watch/scripts/transcribe.py — Parses VTT files into structured transcript segments
• skills/watch/scripts/setup.py — Configures API credentials; referenced when Whisper is unavailable

Summary

• Claude-Video prioritizes embedded captions via fetch_captions in download.py for zero-cost transcript acquisition
• Whisper provides automatic fallback when has_audio is true and API credentials are configured, triggered through transcribe_video
• Silent videos bypass transcription entirely—no errors, just frame extraction
• Missing configurations trigger actionable guidance pointing users to setup.py
• All outcomes are clearly reported in the generated markdown with context-appropriate messaging

Frequently Asked Questions

Does Claude-Video require a Whisper API key to process any video?

No. Videos with embedded captions (most YouTube content) process without any speech-to-text API. Whisper is only invoked when native subtitles are absent and you haven't passed --no-whisper. Without an API key, the tool degrades to frame-only analysis with a clear notice.

What happens if a video has no audio track at all?

The meta.get("has_audio") check in watch.py line 39 detects this condition. Claude-Video skips transcription entirely and generates a frames-only report with the message "No transcript available – proceed with frames only…"

Can I force Claude-Video to fail if no transcript is available?

Yes. Use the --detail transcript flag to enter transcript-only mode. If neither captions nor Whisper can produce text, the script outputs "No transcript available at transcript detail…" and exits without generating a frame analysis.

Which Whisper backends does Claude-Video support?

According to whisper.py, the tool supports both Groq and OpenAI Whisper implementations. The load_api_key function automatically selects the appropriate backend based on configured credentials in your environment.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}