# How Claude-Video Handles Videos Without Captions or Transcripts: A Deep Dive into the Fallback Pipeline

> Discover how Claude-Video manages videos lacking captions or transcripts. Explore its fallback pipeline from embedded captions to automatic Whisper transcripts and frame extraction.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-07-10

---

**Claude-Video implements a three-tier fallback strategy that first attempts to retrieve embedded captions, then automatically generates transcripts using Whisper speech-to-text, and finally falls back to frame-only extraction with clear user notifications when neither option is available.**

When processing videos that lack native captions, the `bradautomates/claude-video` repository employs a robust decision tree to ensure maximum content extraction. This open-source tool located in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) prioritizes transcript availability through multiple pathways before gracefully degrading to visual analysis alone.

## Tier 1: Retrieving Embedded VTT Captions

The first attempt to handle videos without transcripts occurs before any video download completes. In [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) (lines 97-105), the script calls `fetch_captions` from [`download.py`](https://github.com/bradautomates/claude-video/blob/main/download.py) to query for available VTT subtitle files without pulling the actual video content.

If the source platform hosts embedded subtitles, `parse_vtt` immediately converts these caption files into structured transcript segments. This lightweight approach avoids unnecessary processing when textual data already exists on the server.

## Tier 2: On-Demand Whisper Speech-to-Text

When no embedded captions are available, Claude-Video automatically invokes **Whisper speech-to-text** provided the user has not disabled the feature with the `--no-whisper` flag. The logic in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) (lines 39-52) performs the following sequence:

1. **Audio verification** – Checks `meta.get("has_audio")` to confirm the video contains an audio stream
2. **Backend initialization** – Calls `load_api_key` to configure the appropriate Whisper backend (Groq or OpenAI)
3. **Transcription execution** – Runs `transcribe_video` to generate a complete transcript on-the-fly

This fallback ensures that spoken content remains accessible even when publishers do not provide captions.

## Tier 3: Graceful Degradation to Frame Extraction

When transcript generation is impossible, Claude-Video continues processing rather than failing. The tool handles two specific edge cases in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py):

**Missing Whisper Configuration** – If the API key is unavailable or the user specified `--no-whisper`, the script prints a setup hint directing users to configure credentials via [`setup.py`](https://github.com/bradautomates/claude-video/blob/main/setup.py) (lines 55-63), then proceeds to extract frames.

**No Audio Stream** – If `meta.get("has_audio")` returns false, the script skips transcription entirely and proceeds with frame extraction only (lines 65-66).

## User Feedback and Reporting Logic

The generated markdown report clearly communicates the transcript status through three distinct messages defined in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py):

- **Successful transcription** – Displays the actual transcript text when either captions or Whisper succeed
- **Frames-only mode** – Shows *"No transcript available – proceed with frames only…"* when both caption retrieval and Whisper fail (lines 77-84)
- **Transcript-only detail failure** – Displays *"No transcript available at transcript detail…"* when the user explicitly requests `--detail transcript` but no textual content can be produced (lines 68-73)

## Practical Usage Examples

Control the fallback behavior using these command-line patterns:

```bash

# Default behavior: attempts captions first, then Whisper if needed

watch https://www.youtube.com/watch?v=example

# Disable Whisper fallback – forces caption-only or frames-only mode

watch https://www.youtube.com/watch?v=example --no-whisper

# Force transcript-only mode (no frames). Warns if captions/Whisper unavailable

watch https://www.youtube.com/watch?v=example --detail transcript

```

## Key Implementation Files

The transcript acquisition pipeline relies on modular components across the repository:

- **[`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py)** – Orchestrates the entire workflow, deciding between caption retrieval, Whisper invocation, or frame extraction
- **[`skills/watch/scripts/download.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/download.py)** – Handles `fetch_captions` to pull VTT subtitles without downloading video files
- **[`skills/watch/scripts/whisper.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py)** – Manages backend selection and runs `transcribe_video` for speech-to-text conversion
- **[`skills/watch/scripts/transcribe.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/transcribe.py)** – Parses VTT files into transcript segments
- **[`skills/watch/scripts/setup.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/setup.py)** – Configures Whisper API keys when users need to enable speech-to-text

## Summary

- Claude-Video attempts **embedded caption retrieval** first via `fetch_captions` and `parse_vtt` before processing video bytes
- If captions are absent, it **automatically falls back to Whisper** speech-to-text using `transcribe_video` and `load_api_key`
- Users can **disable Whisper** with `--no-whisper` to force caption-only or frames-only processing
- Videos **without audio streams** skip transcription entirely and proceed with frame extraction
- The tool provides **clear status messages** in the final report indicating whether transcripts, frames, or both are available

## Frequently Asked Questions

### What happens if I don't have a Whisper API key configured?

If no API key is present and the video lacks captions, Claude-Video prints a hint directing you to run the setup script to configure Whisper credentials, then proceeds with frame extraction only. The final report will display *"No transcript available – proceed with frames only…"* as implemented in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) lines 77-84.

### Can I force Claude-Video to skip Whisper and use only captions?

Yes. Pass the `--no-whisper` flag when running the `watch` command. This disables the speech-to-text fallback, allowing the tool to proceed directly to frame extraction if no embedded captions are found.

### How does the tool handle videos without any audio track?

When `meta.get("has_audio")` returns false, Claude-Video skips the transcription workflow entirely and proceeds with frame extraction only. This check occurs in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) lines 65-66, ensuring the tool does not attempt to process silent videos.

### What is the difference between --no-whisper and --detail transcript?

The `--no-whisper` flag disables the speech-to-text backend entirely, allowing frame extraction when captions are unavailable. The `--detail transcript` flag forces the tool to output only transcript content without frames; if neither captions nor Whisper can produce text, it displays *"No transcript available at transcript detail…"* and exits without generating frames.