How Audio Extraction Facilitates Whisper Transcription in Claude Video

Audio extraction converts video files into compact 16kHz mono MP3s, enabling chunked uploads to Whisper APIs while preserving timeline accuracy through segment timestamp shifting.

The bradautomates/claude-video repository automates video transcription by first isolating audio tracks before sending them to AI speech recognition services. This audio extraction step is essential because it reduces file sizes to meet strict API upload limits while maintaining the temporal alignment needed for accurate subtitle generation. By implementing a specialized pipeline in skills/watch/scripts/whisper.py, the project ensures that even lengthy videos can be processed efficiently by OpenAI or Groq's Whisper endpoints.

The Audio Extraction Pipeline

Extracting Compact Audio with FFmpeg

The process begins in skills/watch/scripts/whisper.py with the extract_audio() function, which invokes FFmpeg to strip the audio track from the source video. The extraction applies specific parameters to minimize file size while preserving speech clarity: output is set to a mono 16kHz MP3 encoded at approximately 64kbps, resulting in roughly 480kB per minute of audio. This compression is critical because the resulting file must stay under the MAX_UPLOAD_BYTES threshold (24 MiB) enforced by both Whisper backend providers.

Source: extract_audio in [skills/watch/scripts/whisper.py](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py).

Handling Large Files Through Chunking

When transcribe_video() detects that the extracted audio exceeds the upload limit, it triggers an intelligent chunking strategy. The function first calculates a chunking plan using plan_chunks() and then physically splits the audio via split_audio(). This division ensures that each segment remains small enough for the API while preserving enough context for accurate transcription. The chunking logic accounts for the specific duration limits of the target backend (Groq or OpenAI).

Source: transcribe_video, plan_chunks, and split_audio in [skills/watch/scripts/whisper.py](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py).

Uploading to Whisper APIs

Building Multipart Requests

Each audio chunk is uploaded through the _post_whisper() helper, which constructs a multipart/form-data payload using _build_multipart(). The function dynamically selects between GROQ_ENDPOINT and OPENAI_ENDPOINT based on configuration, packaging the audio bytes with the appropriate model parameters. After receiving the JSON response, _segments_from_response() normalizes the output into a uniform segment structure regardless of which backend processed the audio.

Source: _post_whisper, _build_multipart, and _segments_from_response in [skills/watch/scripts/whisper.py](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py).

Reassembling Timestamps with shift_segments

Because Whisper returns timestamps relative to the start of each individual chunk, the pipeline must reconstruct the original timeline. The shift_segments() function offsets each segment's start and end times by the chunk's original position in the video. This adjustment ensures that the final transcript aligns perfectly with the source video's timeline, enabling accurate subtitle synchronization and searchable timestamps.

Source: shift_segments in [skills/watch/scripts/whisper.py](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py).

Integration with the Watch Workflow

The orchestration layer in skills/watch/scripts/watch.py imports transcribe_video and invokes it after downloading the target video. This module handles the complete workflow from source acquisition through final transcript formatting, managing the temporary audio files and cleaning up resources after transcription completes. The seamless integration allows users to process videos via a single command while the complexity of audio extraction and API chunking remains hidden.

Source: Import and execution flow in [skills/watch/scripts/watch.py](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py).

Practical Implementation

To transcribe a video using the Claude Video pipeline:

from skills.watch.scripts.whisper import transcribe_video
from pathlib import Path

# Path to your video file

video_path = Path("lecture.mp4")
audio_path = Path("temp_audio.mp3")

# Extract audio, chunk if necessary, and transcribe

segments, backend_used = transcribe_video(video_path, audio_path)

print(f"Transcribed using: {backend_used}")
for segment in segments:
    print(f"[{segment['start']:.2f}s - {segment['end']:.2f}s] {segment['text']}")

For command-line usage, invoke the watch script directly:

python -m skills.watch.scripts.watch https://example.com/video.mp4

# Automatically downloads, extracts audio, and outputs the transcript

Summary

• FFmpeg extraction in extract_audio() creates a 16kHz mono MP3 at ~64kbps, reducing video files to approximately 480kB per minute to satisfy upload constraints.
• Size validation in transcribe_video() triggers plan_chunks() and split_audio() when audio exceeds 24 MiB, enabling processing of videos of any length.
• API abstraction via _post_whisper() supports both Groq and OpenAI Whisper endpoints through standardized multipart uploads.
• Timeline preservation through shift_segments() reassembles chunk-relative timestamps into absolute video timestamps for accurate alignment.
• Workflow integration in watch.py provides a unified interface that orchestrates download, extraction, and transcription.

Frequently Asked Questions

Why does Claude Video convert video to MP3 before transcription?

Converting to a 16kHz mono MP3 reduces the data payload by roughly 90% compared to uncompressed video while preserving speech intelligibility. This compression ensures the file stays under the 25 MB upload limits imposed by Whisper API providers, eliminating transfer failures and reducing latency.

How does the system handle videos longer than 30 minutes?

When transcribe_video() detects that the extracted audio exceeds MAX_UPLOAD_BYTES (24 MiB), it calculates an optimal chunking strategy using plan_chunks(). The audio is then split into smaller segments that fit within API limits, transcribed in parallel or sequence, and reassembled using shift_segments() to maintain accurate timestamps.

Will the transcript timestamps match the original video timeline?

Yes. Although each audio chunk is transcribed independently with timestamps starting at zero, the shift_segments() function offsets every segment by the chunk's original position in the video. This produces a continuous transcript where timestamps align precisely with the source video's playback position.

Which Whisper backends does Claude Video support?

The repository supports both Groq and OpenAI Whisper endpoints. The _post_whisper() function automatically constructs the appropriate multipart request and headers for the selected backend, allowing users to switch providers by changing configuration variables without modifying the transcription logic.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}