How Claude-Video Handles Videos With No Captions and No Audio Stream

Claude-Video attempts to retrieve embedded captions first, falls back to Whisper speech-to-text if audio exists, and gracefully degrades to frame-only extraction when neither is available, clearly reporting the missing transcript in the final markdown output.

When processing video content that lacks subtitles or spoken audio, the bradautomates/claude-video repository implements a robust fallback chain to maximize usability. Understanding how Claude-Video handles videos with no captions and no audio stream reveals a carefully designed pipeline that prioritizes embedded metadata, attempts AI transcription when possible, and continues with visual analysis when necessary.

The Layered Transcript Acquisition Strategy

Claude-Video follows a strict hierarchy for obtaining textual representations of video content, orchestrated primarily within skills/watch/scripts/watch.py.

Attempting to Retrieve Embedded Captions

When provided with a URL, the system first calls fetch_captions via download.py to download available VTT subtitles without retrieving the full video file (lines 97-105). If subtitles exist, parse_vtt processes them into transcript segments immediately (lines 101-104), bypassing the need for speech-to-text entirely.

Fallback to Whisper Speech-to-Text

If no captions are present and the user has not disabled Whisper via --no-whisper, Claude-Video checks meta.get("has_audio") to verify an audio stream exists. When audio is available, the system loads the appropriate Whisper backend and API key through load_api_key, then executes transcribe_video to generate a transcript on-the-fly (lines 39-52). This functionality leverages the whisper.py module to interface with Groq or OpenAI APIs.

Graceful Degradation When Neither Is Possible

When captions are absent and Whisper cannot run—either due to a missing API key, the --no-whisper flag, or the absence of an audio track—the system adapts rather than failing.

If the video lacks an audio stream entirely, Claude-Video skips transcription and proceeds with frame extraction only (lines 65-66). When Whisper is unavailable due to configuration issues, the script outputs a helpful hint directing users to run setup.py to configure their API key (lines 55-63).

User Feedback and Final Reporting

The generated markdown report clearly communicates the transcript status through specific messages. When both captions and Whisper fail, the report displays "No transcript available – proceed with frames only…" (lines 77-84). If the user explicitly requested transcript-only detail via --detail transcript but no transcription method succeeded, the report shows "No transcript available at transcript detail…" (lines 68-73).

Practical Usage Examples

The following commands demonstrate how to control this behavior from the command line:

# Basic usage – attempts captions first, then Whisper if needed

watch https://www.youtube.com/watch?v=example

# Disable Whisper fallback – useful when you know no API key is configured

watch https://www.youtube.com/watch?v=example --no-whisper

# Force transcript-only mode (no frames). If captions or Whisper are missing, you’ll see a warning.

watch https://www.youtube.com/watch?v=example --detail transcript

Running these commands on videos without subtitles triggers the Whisper fallback (if configured) or produces a frames-only report with a clear notice regarding the missing transcript.

Summary

• Caption-first approach: Claude-Video prioritizes embedded VTT subtitles via fetch_captions in download.py before attempting any audio processing.
• Conditional Whisper fallback: The system only invokes transcribe_video from whisper.py when audio streams are detected and the user has not specified --no-whisper.
• Silent video handling: Videos without audio tracks automatically bypass transcription and proceed to frame extraction.
• Transparent reporting: Users receive clear status messages in the final markdown output indicating whether transcripts were generated from captions, Whisper, or are unavailable.
• Configuration guidance: When Whisper is unavailable due to missing API keys, the system directs users to setup.py for configuration.

Frequently Asked Questions

What happens if a video has no captions and I haven't configured a Whisper API key?

Claude-Video checks for the presence of a Whisper API key via load_api_key. If the key is missing and no embedded captions exist, the script prints a helpful hint directing you to run setup.py to configure Whisper (lines 55-63), then continues with frame extraction while reporting that no transcript is available.

Does Claude-Video download the entire video just to check for captions?

No. The fetch_captions function in download.py downloads only the VTT subtitle files without retrieving the video itself, making the caption check lightweight and fast (lines 97-105). The full video is only processed if caption retrieval fails and frame extraction or Whisper transcription is required.

Can I force Claude-Video to skip audio transcription entirely?

Yes. Passing the --no-whisper flag disables the Whisper fallback, forcing the system to rely solely on embedded captions processed by parse_vtt or transcribe.py. If no captions are found, the tool proceeds directly to frame extraction and reports the absence of transcripts.

How does the tool handle videos that are completely silent (no audio stream)?

When metadata indicates has_audio is false, Claude-Video skips the transcription workflow entirely (lines 65-66). It proceeds with frame extraction and includes the message "No transcript available – proceed with frames only…" in the final report, ensuring users understand why textual analysis is unavailable.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}