# Transcript Fallback System in Claude Video: Automatic Whisper Transcription When Captions Are Missing

> Discover Claude Video's transcript fallback system. It uses Whisper AI for automatic transcription when captions are missing, converting video audio to text for easy access.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: how-to-guide
- Published: 2026-08-01

---

**When yt-dlp cannot retrieve native captions, Claude Video automatically switches to OpenAI or Groq Whisper transcription, extracting audio from the video and converting speech to text in VTT-compatible segments.**

The transcript fallback system in the `bradautomates/claude-video` repository ensures users always receive textual content from videos, even when subtitle tracks are unavailable. According to the source code in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py), the system first attempts to fetch native captions via yt-dlp, then seamlessly invokes a Whisper-based transcription pipeline when those captions are missing. This fallback requires either a Groq or OpenAI API key configured in `~/.config/watch/.env` or a local `.env` file.

## How the System Detects Missing Captions

The fallback mechanism begins immediately after the initial caption download attempt. In [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py), the `fetch_captions` function attempts to retrieve subtitles via yt-dlp and stores the result in a download dictionary.

The code checks whether `dl["subtitle_path"]` exists in the returned data structure. If this key is absent, the transcript variables remain empty, signaling that native captions are unavailable and triggering the Whisper fallback pathway.

## Configuring the Whisper Transcription Backend

The Whisper fallback is enabled by default, but users can control its behavior through command-line flags and configuration files.

### Disabling the Fallback

To bypass audio transcription entirely and proceed with visual analysis only, use the `--no-whisper` flag implemented in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) at lines 53-56:

```bash
watch https://example.com/video.mp4 --no-whisper

```

### API Key Resolution

The `load_api_key` function in **[`skills/watch/scripts/whisper.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py)** determines which backend to use by checking for valid credentials in the following order:

1. **Groq** — preferred provider if `GROQ_API_KEY` is present
2. **OpenAI** — fallback provider if `OPENAI_API_KEY` is present and Groq is unavailable

The function searches first in environment variables, then in `~/.config/watch/.env`, and finally in a local `.env` file in the current directory.

### Initial Setup

To scaffold the configuration file, run [`skills/watch/scripts/setup.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/setup.py), which creates the `.env` file and prints configuration instructions:

```bash
python3 skills/watch/scripts/setup.py

```

Edit the created file at `~/.config/watch/.env` to add your preferred key:

```bash
GROQ_API_KEY=your_groq_key

# OR

OPENAI_API_KEY=your_openai_key

```

## The Audio-to-Text Transcription Pipeline

When captions are missing and a valid API key is detected, [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) invokes the transcription pipeline at lines 252-263.

The `transcribe_video` function in [`whisper.py`](https://github.com/bradautomates/claude-video/blob/main/whisper.py) executes the following steps:

1. **Audio Extraction** — extracts the audio track from the video file
2. **Chunk Planning** — splits the audio into appropriately-sized segments to respect API limits
3. **Parallel Upload** — uploads each chunk to the chosen Whisper endpoint (Groq or OpenAI)
4. **Segment Assembly** — reconstructs the returned segments into VTT-compatible format using helpers from [`skills/watch/scripts/transcribe.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/transcribe.py), including `filter_range` and `format_transcript`

The assembled transcript matches the same shape as native VTT subtitles, ensuring downstream processors handle caption-derived and Whisper-derived text identically.

## Graceful Degradation to Frames-Only Mode

If the Whisper call fails due to network errors, invalid API keys, or missing configuration, the system degrades gracefully. In [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) at lines 265-283, the error handling prints a helpful hint explaining the transcription failure, then continues execution with frames-only output.

This ensures the tool remains functional for visual analysis even when audio transcription is impossible. To force this behavior proactively, users can combine the `--no-whisper` flag with their command.

## Summary

- The transcript fallback system activates when `dl["subtitle_path"]` is absent after the initial caption fetch attempt
- Whisper transcription is enabled by default but can be disabled via the `--no-whisper` flag (lines 53-56 in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py))
- The `load_api_key` function in [`whisper.py`](https://github.com/bradautomates/claude-video/blob/main/whisper.py) prefers Groq over OpenAI when resolving API credentials
- Audio processing occurs in `transcribe_video` (lines 252-263 in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py)), which chunks audio and formats output using [`transcribe.py`](https://github.com/bradautomates/claude-video/blob/main/transcribe.py) utilities
- When transcription fails or no API key exists, the system falls back to frames-only output (lines 265-283 in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py))

## Frequently Asked Questions

### What triggers the transcript fallback system in Claude Video?

The fallback triggers when the `fetch_captions` function fails to populate `dl["subtitle_path"]` in the download dictionary, indicating that yt-dlp could not retrieve native subtitles for the video. At this point, the script automatically attempts Whisper transcription if the user has not disabled it with `--no-whisper`.

### How do I disable the Whisper fallback and use only visual frames?

Pass the `--no-whisper` flag when invoking the watch command. This bypasses the audio extraction and API calls entirely, forcing the system to proceed directly to frame extraction and visual analysis without attempting speech-to-text transcription.

### Which Whisper API provider does Claude Video prefer?

According to the `load_api_key` implementation in [`whisper.py`](https://github.com/bradautomates/claude-video/blob/main/whisper.py), the system prefers **Groq** over OpenAI. It checks for `GROQ_API_KEY` first in environment variables and configuration files, falling back to `OPENAI_API_KEY` only if Groq credentials are unavailable.

### What happens if I don't have a Whisper API key configured?

If no valid API key is found in `~/.config/watch/.env`, local `.env`, or environment variables, or if the API call fails, the script catches the error in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) (lines 265-283), prints a helpful message explaining the missing transcription, and continues with frames-only output. The video processing completes successfully using only visual information.