How Claude-Video Handles Videos Without Audio Streams: Metadata Detection and Conditional Processing

The claude-video script uses ffprobe metadata detection to identify silent videos and conditionally skips Whisper transcription while continuing frame extraction and reporting.

The bradautomates/claude-video repository provides a Python-based video processing pipeline that downloads, analyzes, and transcribes video content. When processing videos without audio streams, the script implements specific detection logic in watch.py and frames.py to avoid transcription errors while maintaining full functionality for visual analysis.

Detecting Audio Streams with ffprobe

The foundation of silent video handling relies on metadata inspection before processing begins.

Metadata Extraction in frames.py

According to the source code, the get_metadata() function in skills/watch/scripts/frames.py inspects video containers using ffprobe and returns a dictionary containing a has_audio key. This boolean value is set to True when an audio stream is present and False otherwise (lines 11-19).

Conditional Processing Logic in watch.py

The main orchestrator skills/watch/scripts/watch.py uses this metadata to make intelligent decisions about transcription workflows.

Transcript-Only Mode Optimization

If the user requests only a transcript using the --detail transcript flag and the video contains no audio, the script skips the download entirely. The audio_only flag is only set to True when the metadata confirms an audio stream exists.

Whisper Fallback Prevention

When the script attempts to obtain a transcript, it first checks for available subtitles. If none are found, the Whisper fallback in skills/watch/scripts/whisper.py is invoked only when meta.get("has_audio") returns true. According to lines 265-267 in watch.py, if no audio stream exists, the code executes:

elif not transcript_segments and video_path and not meta.get("has_audio"):
    print("[watch] no audio stream found — proceeding without transcription", file=sys.stderr)

This logs the absence of audio to stderr and continues execution without attempting transcription.

Continued Frame Processing

Downstream processing proceeds normally regardless of audio status. The frame extraction logic does not require an audio track, so the script generates the final report containing extracted frames alongside a "none available" transcript status.

Practical Command-Line Examples

When processing a silent MP4 file, the script provides clear feedback:

$ watch https://example.com/silent-video.mp4 --detail balanced
[watch] checking metadata/captions via yt-dlp…
[watch] downloading video via yt-dlp…
[watch] no audio stream found — proceeding without transcription

The output report includes:

- **Transcript:** none available

If requesting transcript-only mode on a silent video:

$ watch https://example.com/silent-video.mp4 --detail transcript
[watch] checking metadata/captions via yt-dlp…
[watch] no audio stream found — proceeding without transcription

Summary

• Metadata detection occurs via get_metadata() in frames.py using ffprobe to set a has_audio boolean (lines 11-19).
• Conditional logic in watch.py checks meta.get("has_audio") before invoking Whisper transcription (lines 265-267).
• Silent videos trigger a stderr message and skip transcription while continuing frame extraction.
• Transcript-only requests for silent videos skip unnecessary downloads entirely.

Frequently Asked Questions

How does claude-video detect the absence of audio streams?

The script calls get_metadata() from skills/watch/scripts/frames.py (lines 11-19), which uses ffprobe to analyze the video container and returns a has_audio key set to False when no audio stream is present.

Will the script attempt to transcribe a video without audio?

No. According to lines 265-267 in skills/watch/scripts/watch.py, the script checks meta.get("has_audio") before attempting Whisper transcription. If false, it prints a message to stderr and proceeds without transcription.

Does frame extraction work on videos without audio?

Yes. Frame extraction operates independently of audio streams. The script processes visual content normally and includes the extracted frames in the final report, marking the transcript as "none available."

What happens when requesting --detail transcript for a silent video?

The script skips the download entirely because the audio_only flag is only set when has_audio is true. It then reports that no audio stream was found and exits without attempting frame extraction or transcription.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}