How Claude-Video Uses Whisper for Video Transcription

Claude-Video's /watch skill delegates audio transcription to OpenAI's Whisper API via a dedicated whisper.py module that handles authentication, streaming, and response parsing when built-in captions are unavailable.

The bradautomates/claude-video repository implements a modular video analysis system where the watch skill extracts frames and transcripts from video URLs or local files. When processing audio content, the system intelligently routes transcription requests through a dedicated pipeline that prioritizes existing captions but falls back to Whisper for maximum accuracy.

The Transcription Pipeline Architecture

The transcription workflow follows a clear separation of concerns across three core modules in skills/watch/scripts/. This design keeps the host-agnostic skill portable across Claude Code, Codex, and Cursor environments.

Entry Point and Orchestration (watch.py)

The skills/watch/scripts/watch.py file serves as the primary orchestrator. It coordinates video download, frame extraction, and transcription by invoking helper modules in sequence. When a user triggers the /watch slash command, this script first calls download.py to retrieve the video asset, then frames.py for visual analysis, and finally transcribe.py to handle audio-to-text conversion.

Caption vs. Whisper Decision Logic (transcribe.py)

Located at skills/watch/scripts/transcribe.py, this module implements the decision logic for transcription sources. It first inspects the video container for embedded subtitle tracks. If captions exist and the user has not requested forced transcription, it extracts them directly. Otherwise, it imports and executes skills/watch/scripts/whisper.py to generate a fresh transcript via the OpenAI API.

Whisper API Integration (whisper.py)

The skills/watch/scripts/whisper.py module contains the actual Whisper implementation. It performs three critical operations:

• Authentication: Loads the OPENAI_API_KEY from a user-specific .env file stored at ~/.config/watch/.env via skills/watch/scripts/config.py
• Request Handling: Streams the video's audio track to https://api.openai.com/v1/audio/transcriptions using the whisper-1 model
• Response Parsing: Extracts the plain-text transcription from the JSON response and returns it to transcribe.py

The function signature follows this pattern:

# skills/watch/scripts/whisper.py

def transcribe_with_whisper(audio_path: str) -> str:
    """
    Sends audio file to Whisper API and returns transcription text.
    """
    # Loads API key from config

    # POSTs to OpenAI audio transcriptions endpoint

    # Returns extracted text string

Configuration and Authentication

The system uses a host-agnostic configuration pattern to keep API keys secure. The skills/watch/scripts/config.py module resolves the path to ~/.config/watch/.env, ensuring the Whisper client can authenticate without relying on global environment variables. This allows multiple users on shared systems to maintain separate OpenAI credentials.

Required .env format:

# ~/.config/watch/.env

OPENAI_API_KEY=sk-your-key-here

Practical Usage Examples

You can invoke Whisper transcription through the CLI slash command or programmatically via the Python API.

Basic CLI Usage

Install the skill globally and process any video URL:

npx skills add bradautomates/claude-video -g
watch "https://www.youtube.com/watch?v=dQw4w9WgXcQ"

The system automatically detects missing captions and routes the audio to Whisper.

Forcing Whisper Transcription in Python

To bypass embedded captions and force API-based transcription:

from skills.watch.scripts.watch import run_watch

result = run_watch(
    url="https://www.youtube.com/watch?v=dQw4w9WgXcQ",
    force_whisper=True,  # Explicitly invokes whisper.py

)

print(result["transcript"])  # Output: Whisper-generated text

Direct Module Access

For advanced use cases requiring direct audio file processing:

from skills.watch.scripts.whisper import transcribe_with_whisper

audio_file = "extracted_audio.wav"
transcription = transcribe_with_whisper(audio_file)
print(f"Transcribed {len(transcription)} characters")

Summary

• Modular Architecture: watch.py orchestrates the pipeline while transcribe.py decides between caption extraction and Whisper API calls
• Secure Authentication: API keys reside in ~/.config/watch/.env, loaded by config.py to keep credentials host-agnostic
• Fallback Logic: The system prioritizes existing captions but seamlessly falls back to skills/watch/scripts/whisper.py when needed
• Flexible Interface: Use the /watch slash command, the run_watch() Python API with force_whisper=True, or call transcribe_with_whisper() directly

Frequently Asked Questions

Where does Claude-Video store the OpenAI API key for Whisper?

The system reads the OPENAI_API_KEY from a local .env file located at ~/.config/watch/.env. The skills/watch/scripts/config.py module resolves this path dynamically, allowing the skill to operate across different hosts without requiring global environment variables.

Can I force Whisper transcription even if a video has embedded captions?

Yes. Pass force_whisper=True to the run_watch() function in skills/watch/scripts/watch.py. This parameter instructs transcribe.py to skip caption extraction and immediately invoke whisper.py for API-based transcription.

What Whisper model does Claude-Video use?

According to the implementation in skills/watch/scripts/whisper.py, the module targets the whisper-1 endpoint at https://api.openai.com/v1/audio/transcriptions. This corresponds to OpenAI's standard large-v2 model available through their API.

Is the Whisper integration host-specific?

No. The whisper.py module is designed to be host-agnostic. By storing configuration in a user-specific .env file rather than relying on system-wide environment variables, the skill functions identically across Claude Code, Codex, Cursor, and local CLI installations.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}