How the Watch Skill Handles Videos Without Captions or Audio

The watch skill attempts to fetch existing captions first, falls back to OpenAI Whisper transcription when audio is present and API keys are configured, and gracefully degrades to frame-only visual analysis when neither captions nor audio are available.

The watch skill in the bradautomates/claude-video repository implements a resilient, multi-layered strategy for video transcription that accommodates missing subtitle files and silent videos. According to the source code in skills/watch/scripts/watch.py, the skill orchestrates caption downloads, audio transcription, and frame extraction to ensure useful output even when video metadata lacks text or audio tracks.

The Layered Transcription Hierarchy

The skill processes videos through a strict priority system to obtain transcript data while minimizing unnecessary downloads.

Attempting Caption Downloads First

When processing a URL, the skill calls fetch_captions at line 99 in skills/watch/scripts/watch.py to request subtitle files via yt-dlp. If subtitles exist, the parse_vtt function from skills/watch/scripts/transcribe.py (line 24) parses the WebVTT format into timestamped transcript segments. If dl.get("subtitle_path") returns falsy, the transcript_segments list remains empty and the skill proceeds to the next layer.

Skipping Video Downloads for Transcript-Only Mode

For efficiency, the skill checks whether a video download is necessary when the user requests --detail transcript. If no caption file is found and no explicit timestamp cues were supplied, the script sets video_path = None at lines 112-113, preventing unnecessary bandwidth usage when audio extraction would not yield usable transcript data.

Fallback to Whisper Audio Transcription

When captions are unavailable but the video contains an audio track, the skill attempts AI-based transcription.

The Whisper Activation Check

After frame-extraction logic, skills/watch/scripts/watch.py evaluates the condition at lines 39-41:

if not transcript_segments and not args.no_whisper and video_path and meta.get("has_audio"):

This ensures Whisper runs only when no existing transcripts exist, the user has not disabled Whisper via --no-whisper, a video path is available, and metadata confirms has_audio is True.

Executing the Transcription Pipeline

When conditions are met, the skill loads the Whisper backend using load_api_key (line 42) and calls transcribe_video (lines 43-48) to generate transcript segments from the extracted audio. Upon success, the transcript source is annotated as whisper (backend) at line 51, ensuring traceability in the final report.

Handling Missing API Configuration

If no API key is configured or the user explicitly disabled Whisper, the skill prints a configuration hint at lines 55-64, directing users to run setup.py to install a Whisper key before retrying.

Processing Videos Without Audio Streams

For silent videos or media files lacking audio tracks, the skill bypasses transcription entirely. At lines 65-66 in skills/watch/scripts/watch.py, the code checks:

elif not transcript_segments and video_path and not meta.get("has_audio"):

When has_audio is False, the skill logs "no audio stream found — proceeding without transcription" and continues with frame extraction only, ensuring the process completes without errors on silent animations or mute recordings.

Final Report Generation

Regardless of transcription success, the skill generates a markdown report containing a "Transcript" section (lines 68-84). If no transcript could be constructed, the report includes a user-friendly explanation of why transcription was skipped (missing captions, disabled Whisper, or absent audio), alongside any successfully extracted frames. This graceful degradation ensures users receive visual analysis even when text transcription is impossible.

Usage Examples

The following commands demonstrate how the watch skill handles different video conditions:

# Video with existing captions - no Whisper needed

watch https://www.youtube.com/watch?v=example123

# Video without captions but with audio - requires Whisper API key

watch https://example.com/lecture.mp4

# Explicitly disable Whisper fallback

watch https://example.com/lecture.mp4 --no-whisper

# Silent video or animation - produces frames-only output

watch https://example.com/silent-demo.mp4

Key Implementation Files

The transcription logic is distributed across modular components:

• skills/watch/scripts/watch.py: Main orchestration; handles caption download, Whisper fallback logic, and frame extraction workflow.
• skills/watch/scripts/download.py: Interfaces with yt-dlp to fetch video files, audio tracks, and subtitle files.
• skills/watch/scripts/transcribe.py: Parses WebVTT caption files into clean transcript segments via parse_vtt.
• skills/watch/scripts/whisper.py: Manages API key loading and executes Whisper transcription services.
• skills/watch/scripts/config.py: Provides default configuration values including detail levels affecting download behavior.

Summary

• Caption-first approach: The skill always attempts to download existing subtitles before invoking expensive transcription services.
• Intelligent download skipping: When only transcripts are requested and none exist, the skill avoids downloading video files unnecessarily.
• Whisper fallback: Audio tracks are transcribed via OpenAI Whisper only when captions are missing, audio exists, and API keys are configured.
• Silent video support: Videos without audio streams bypass transcription entirely and proceed with frame extraction.
• Graceful degradation: Every execution produces a markdown report explaining transcription status, ensuring transparency when captions or audio are unavailable.

Frequently Asked Questions

What happens if a video has no captions and I don't have a Whisper API key configured?

If no subtitles are found and Whisper is unavailable (missing API key or disabled via --no-whisper), the skill skips transcription and generates a frames-only report. According to lines 55-64 in skills/watch/scripts/watch.py, the skill prints a hint directing you to run setup.py to configure a Whisper key, then continues processing visual frames without failing.

Does the watch skill download the entire video if I only need a transcript but no captions exist?

No. When you specify --detail transcript and no captions are found, the skill sets video_path = None at lines 112-113, preventing the video download entirely unless timestamp cues or frame extraction are explicitly required.

How does the skill detect whether a video has an audio track?

The skill checks the has_audio field in the video metadata dictionary. At lines 39-41 and 65-66 in skills/watch/scripts/watch.py, the code evaluates meta.get("has_audio") to determine whether to attempt Whisper transcription or skip directly to frame extraction.

Can I force the watch skill to ignore captions and use Whisper transcription instead?

No. The skill prioritizes existing captions automatically via fetch_captions (line 99) and only proceeds to Whisper if transcript_segments remains empty. There is currently no command-line flag to bypass available captions in favor of Whisper transcription.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}