# How the Whisper Fallback Process Works in Claude-Video: Groq vs OpenAI

> Understand the Whisper fallback process in claude-video. See how it prioritizes Groq then falls back to OpenAI for seamless transcription.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-07-07

---

**The Whisper fallback process in claude-video automatically prefers Groq when both API keys are present, falling back to OpenAI only when the Groq key is absent, using a unified transcription interface that normalizes responses regardless of backend.**

The claude-video repository implements a resilient **Whisper fallback process** that automatically routes transcription requests to available backends. Located in [`skills/watch/scripts/whisper.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py), this module abstracts the differences between Groq and OpenAI's transcription services, providing a single interface that prioritizes speed and availability.

## How the Backend Selection Works

The module determines which transcription service to use through a hierarchical API key discovery mechanism.

**`load_api_key()`** scans the environment variables (or a per-user `.env` file) for `GROQ_API_KEY` and `OPENAI_API_KEY` on lines 65-71 of [`whisper.py`](https://github.com/bradautomates/claude-video/blob/main/whisper.py). The logic follows this priority:

1. **Groq is preferred** – When both keys are present, the candidate list selects Groq first (lines 68-71).
2. **Single key fallback** – If only one key exists, that backend is selected automatically.
3. **Hard failure** – If neither key is found, the script aborts with a clear error message directing users to the setup helper.

This design ensures that **Groq acts as the primary backend** while OpenAI serves as the transparent fallback.

## Groq vs OpenAI Implementation Details

Once `transcribe_video()` selects a backend, it delegates to `_transcribe_file()`, which constructs backend-specific requests.

**Endpoint and Model Differences**

- **Groq**: Uses `https://api.groq.com/openai/v1/audio/transcriptions` with the `whisper-large-v3` model (defined as `GROQ_MODEL`).
- **OpenAI**: Uses `https://api.openai.com/v1/audio/transcriptions` with the `whisper-1` model (defined as `OPENAI_MODEL`).

**Request Construction Nuances**

Both services receive identical multipart-form payloads created by `_build_multipart()`. However, `_post_whisper()` (lines 44-51) adds a critical distinction:

- **Groq**: Requires a custom User-Agent header (`watch-skill/1.0 (+claude-code; python-urllib)`) to bypass Cloudflare WAF rule 1010.
- **OpenAI**: Uses the default User-Agent without modifications.

## Error Handling and Rate Limiting

The fallback process includes robust retry logic that applies uniformly to both backends.

**Upload Size Management**

The module enforces a **24 MiB maximum upload size** (`MAX_UPLOAD_BYTES`). When `plan_chunks()` detects larger files, it triggers `split_audio()` to create sequential chunks, then processes them via `transcribe_chunks()`.

**Retry Strategy**

Both backends share the same exponential backoff implementation:
- **Maximum attempts**: 4 retries (`MAX_ATTEMPTS`)
- **Rate limit handling**: Additional dedicated retries for HTTP 429 responses (`MAX_429_RETRIES`)
- **Implementation**: Lines 32-38 define constants, while lines 58-87 contain the retry loop in [`whisper.py`](https://github.com/bradautomates/claude-video/blob/main/whisper.py).

## Normalizing Responses Across Backends

Regardless of which service generates the transcript, the module returns a standardized format.

**`_segments_from_response()`** (lines 50-68) converts raw JSON responses into a uniform `{start, end, text}` segment structure. This normalization allows the rest of the claude-video pipeline to treat Groq and OpenAI transcripts identically, eliminating downstream conditional logic.

## Practical Usage Examples

You can leverage the fallback process programmatically or via command line.

**Automatic Backend Selection**

```python
from pathlib import Path
from skills.watch.scripts.whisper import transcribe_video

video_path = "example.mp4"
audio_out = Path("tmp/audio.mp3")
segments, used_backend = transcribe_video(video_path, audio_out)
print(f"Transcribed with {used_backend}: {len(segments)} segments")

```

**Forcing a Specific Backend**

```python

# Force OpenAI even when Groq key is present

segments, used_backend = transcribe_video(
    "example.mp4",
    Path("tmp/audio.mp3"),
    backend="openai"
)

```

**Command-Line Usage**

```bash

# Let the script auto-select

python -m skills.watch.scripts.whisper example.mp4

# Force specific backend

python -m skills.watch.scripts.whisper example.mp4 --backend groq
python -m skills.watch.scripts.whisper example.mp4 --backend openai

```

## Summary

- **Groq is prioritized**: When both `GROQ_API_KEY` and `OPENAI_API_KEY` exist, the Whisper fallback process selects Groq automatically.
- **Identical interface**: Both backends use the same retry logic, chunking strategy, and response normalization via `_segments_from_response()`.
- **Header differentiation**: Groq requires a custom User-Agent to bypass Cloudflare WAF, while OpenAI uses standard headers.
- **Model variation**: Groq uses `whisper-large-v3` (larger model) versus OpenAI's `whisper-1`.
- **Override capability**: Users can force a specific backend using the `--backend` CLI flag or `backend` parameter in Python.

## Frequently Asked Questions

### How does claude-video decide which Whisper backend to use?

The decision occurs in `load_api_key()` within [`skills/watch/scripts/whisper.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py). The function builds a candidate list of available API keys (lines 65-71), preferring Groq when both keys are present. If only one key exists, that backend is selected; if neither exists, the script raises an error with setup instructions.

### What happens if both Groq and OpenAI API keys are configured?

When both `GROQ_API_KEY` and `OPENAI_API_KEY` are present in the environment, the module automatically selects Groq as the primary backend. OpenAI serves as a silent fallback, only activating if the Groq key is removed or invalidated. Users can override this default by passing `backend="openai"` to `transcribe_video()` or using the `--backend openai` CLI flag.

### Why does the Groq backend require a custom User-Agent header?

According to the implementation in `_post_whisper()` (lines 44-51), Groq sits behind Cloudflare, which triggers WAF rule 1010 for requests lacking the custom header `watch-skill/1.0 (+claude-code; python-urllib)`. This header identifies the request as legitimate traffic from the claude-video tool. OpenAI's endpoint does not implement this specific WAF rule, so it accepts the default Python User-Agent.

### How does the module handle audio files larger than 24 MiB?

The `transcribe_video()` function checks file size against `MAX_UPLOAD_BYTES` (24 MiB). For larger files, `plan_chunks()` calculates the necessary splits, `split_audio()` creates the physical chunks, and `transcribe_chunks()` processes each segment sequentially. This chunked approach applies identically to both Groq and OpenAI backends, with the same retry logic applied to each individual chunk.