How the Whisper Fallback Process Works in Claude-Video: Groq vs OpenAI

The Whisper fallback process in claude-video automatically prefers Groq when both API keys are present, falling back to OpenAI only when the Groq key is absent, using a unified transcription interface that normalizes responses regardless of backend.

The claude-video repository implements a resilient Whisper fallback process that automatically routes transcription requests to available backends. Located in skills/watch/scripts/whisper.py, this module abstracts the differences between Groq and OpenAI's transcription services, providing a single interface that prioritizes speed and availability.

How the Backend Selection Works

The module determines which transcription service to use through a hierarchical API key discovery mechanism.

load_api_key() scans the environment variables (or a per-user .env file) for GROQ_API_KEY and OPENAI_API_KEY on lines 65-71 of whisper.py. The logic follows this priority:

1. Groq is preferred – When both keys are present, the candidate list selects Groq first (lines 68-71).
2. Single key fallback – If only one key exists, that backend is selected automatically.
3. Hard failure – If neither key is found, the script aborts with a clear error message directing users to the setup helper.

This design ensures that Groq acts as the primary backend while OpenAI serves as the transparent fallback.

Groq vs OpenAI Implementation Details

Once transcribe_video() selects a backend, it delegates to _transcribe_file(), which constructs backend-specific requests.

Endpoint and Model Differences

• Groq: Uses https://api.groq.com/openai/v1/audio/transcriptions with the whisper-large-v3 model (defined as GROQ_MODEL).
• OpenAI: Uses https://api.openai.com/v1/audio/transcriptions with the whisper-1 model (defined as OPENAI_MODEL).

Request Construction Nuances

Both services receive identical multipart-form payloads created by _build_multipart(). However, _post_whisper() (lines 44-51) adds a critical distinction:

• Groq: Requires a custom User-Agent header (watch-skill/1.0 (+claude-code; python-urllib)) to bypass Cloudflare WAF rule 1010.
• OpenAI: Uses the default User-Agent without modifications.

Error Handling and Rate Limiting

The fallback process includes robust retry logic that applies uniformly to both backends.

Upload Size Management

The module enforces a 24 MiB maximum upload size (MAX_UPLOAD_BYTES). When plan_chunks() detects larger files, it triggers split_audio() to create sequential chunks, then processes them via transcribe_chunks().

Retry Strategy

Both backends share the same exponential backoff implementation:

• Maximum attempts: 4 retries (MAX_ATTEMPTS)
• Rate limit handling: Additional dedicated retries for HTTP 429 responses (MAX_429_RETRIES)
• Implementation: Lines 32-38 define constants, while lines 58-87 contain the retry loop in whisper.py.

Normalizing Responses Across Backends

Regardless of which service generates the transcript, the module returns a standardized format.

_segments_from_response() (lines 50-68) converts raw JSON responses into a uniform {start, end, text} segment structure. This normalization allows the rest of the claude-video pipeline to treat Groq and OpenAI transcripts identically, eliminating downstream conditional logic.

Practical Usage Examples

You can leverage the fallback process programmatically or via command line.

Automatic Backend Selection

from pathlib import Path
from skills.watch.scripts.whisper import transcribe_video

video_path = "example.mp4"
audio_out = Path("tmp/audio.mp3")
segments, used_backend = transcribe_video(video_path, audio_out)
print(f"Transcribed with {used_backend}: {len(segments)} segments")

Forcing a Specific Backend

# Force OpenAI even when Groq key is present

segments, used_backend = transcribe_video(
    "example.mp4",
    Path("tmp/audio.mp3"),
    backend="openai"
)

Command-Line Usage

# Let the script auto-select

python -m skills.watch.scripts.whisper example.mp4

# Force specific backend

python -m skills.watch.scripts.whisper example.mp4 --backend groq
python -m skills.watch.scripts.whisper example.mp4 --backend openai

Summary

• Groq is prioritized: When both GROQ_API_KEY and OPENAI_API_KEY exist, the Whisper fallback process selects Groq automatically.
• Identical interface: Both backends use the same retry logic, chunking strategy, and response normalization via _segments_from_response().
• Header differentiation: Groq requires a custom User-Agent to bypass Cloudflare WAF, while OpenAI uses standard headers.
• Model variation: Groq uses whisper-large-v3 (larger model) versus OpenAI's whisper-1.
• Override capability: Users can force a specific backend using the --backend CLI flag or backend parameter in Python.

Frequently Asked Questions

How does claude-video decide which Whisper backend to use?

The decision occurs in load_api_key() within skills/watch/scripts/whisper.py. The function builds a candidate list of available API keys (lines 65-71), preferring Groq when both keys are present. If only one key exists, that backend is selected; if neither exists, the script raises an error with setup instructions.

What happens if both Groq and OpenAI API keys are configured?

When both GROQ_API_KEY and OPENAI_API_KEY are present in the environment, the module automatically selects Groq as the primary backend. OpenAI serves as a silent fallback, only activating if the Groq key is removed or invalidated. Users can override this default by passing backend="openai" to transcribe_video() or using the --backend openai CLI flag.

Why does the Groq backend require a custom User-Agent header?

According to the implementation in _post_whisper() (lines 44-51), Groq sits behind Cloudflare, which triggers WAF rule 1010 for requests lacking the custom header watch-skill/1.0 (+claude-code; python-urllib). This header identifies the request as legitimate traffic from the claude-video tool. OpenAI's endpoint does not implement this specific WAF rule, so it accepts the default Python User-Agent.

How does the module handle audio files larger than 24 MiB?

The transcribe_video() function checks file size against MAX_UPLOAD_BYTES (24 MiB). For larger files, plan_chunks() calculates the necessary splits, split_audio() creates the physical chunks, and transcribe_chunks() processes each segment sequentially. This chunked approach applies identically to both Groq and OpenAI backends, with the same retry logic applied to each individual chunk.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}