# How Audio Extraction Facilitates Whisper Transcription in Claude Video

> Learn how audio extraction in Claude Video creates MP3s for Whisper transcription. Discover how it preserves timeline accuracy with segment timestamp shifting.

- Repository: [bradautomates/claude-video](https://github.com/bradautomates/claude-video)
- Tags: deep-dive
- Published: 2026-07-11

---

**Audio extraction converts video files into compact 16kHz mono MP3s, enabling chunked uploads to Whisper APIs while preserving timeline accuracy through segment timestamp shifting.**

The `bradautomates/claude-video` repository automates video transcription by first isolating audio tracks before sending them to AI speech recognition services. This audio extraction step is essential because it reduces file sizes to meet strict API upload limits while maintaining the temporal alignment needed for accurate subtitle generation. By implementing a specialized pipeline in [`skills/watch/scripts/whisper.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py), the project ensures that even lengthy videos can be processed efficiently by OpenAI or Groq's Whisper endpoints.

## The Audio Extraction Pipeline

### Extracting Compact Audio with FFmpeg

The process begins in [`skills/watch/scripts/whisper.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py) with the `extract_audio()` function, which invokes **FFmpeg** to strip the audio track from the source video. The extraction applies specific parameters to minimize file size while preserving speech clarity: output is set to a **mono 16kHz MP3** encoded at approximately **64kbps**, resulting in roughly **480kB per minute** of audio. This compression is critical because the resulting file must stay under the `MAX_UPLOAD_BYTES` threshold (24 MiB) enforced by both Whisper backend providers.

*Source:* `extract_audio` in [[`skills/watch/scripts/whisper.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py)](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py).

### Handling Large Files Through Chunking

When `transcribe_video()` detects that the extracted audio exceeds the upload limit, it triggers an intelligent chunking strategy. The function first calculates a chunking plan using `plan_chunks()` and then physically splits the audio via `split_audio()`. This division ensures that each segment remains small enough for the API while preserving enough context for accurate transcription. The chunking logic accounts for the specific duration limits of the target backend (Groq or OpenAI).

*Source:* `transcribe_video`, `plan_chunks`, and `split_audio` in [[`skills/watch/scripts/whisper.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py)](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py).

## Uploading to Whisper APIs

### Building Multipart Requests

Each audio chunk is uploaded through the `_post_whisper()` helper, which constructs a **multipart/form-data** payload using `_build_multipart()`. The function dynamically selects between `GROQ_ENDPOINT` and `OPENAI_ENDPOINT` based on configuration, packaging the audio bytes with the appropriate model parameters. After receiving the JSON response, `_segments_from_response()` normalizes the output into a uniform segment structure regardless of which backend processed the audio.

*Source:* `_post_whisper`, `_build_multipart`, and `_segments_from_response` in [[`skills/watch/scripts/whisper.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py)](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py).

### Reassembling Timestamps with shift_segments

Because Whisper returns timestamps relative to the start of each individual chunk, the pipeline must reconstruct the original timeline. The `shift_segments()` function offsets each segment's start and end times by the chunk's original position in the video. This adjustment ensures that the final transcript aligns perfectly with the source video's timeline, enabling accurate subtitle synchronization and searchable timestamps.

*Source:* `shift_segments` in [[`skills/watch/scripts/whisper.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py)](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/whisper.py).

## Integration with the Watch Workflow

The orchestration layer in [`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py) imports `transcribe_video` and invokes it after downloading the target video. This module handles the complete workflow from source acquisition through final transcript formatting, managing the temporary audio files and cleaning up resources after transcription completes. The seamless integration allows users to process videos via a single command while the complexity of audio extraction and API chunking remains hidden.

*Source:* Import and execution flow in [[`skills/watch/scripts/watch.py`](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py)](https://github.com/bradautomates/claude-video/blob/main/skills/watch/scripts/watch.py).

## Practical Implementation

To transcribe a video using the Claude Video pipeline:

```python
from skills.watch.scripts.whisper import transcribe_video
from pathlib import Path

# Path to your video file

video_path = Path("lecture.mp4")
audio_path = Path("temp_audio.mp3")

# Extract audio, chunk if necessary, and transcribe

segments, backend_used = transcribe_video(video_path, audio_path)

print(f"Transcribed using: {backend_used}")
for segment in segments:
    print(f"[{segment['start']:.2f}s - {segment['end']:.2f}s] {segment['text']}")

```

For command-line usage, invoke the watch script directly:

```bash
python -m skills.watch.scripts.watch https://example.com/video.mp4

# Automatically downloads, extracts audio, and outputs the transcript

```

## Summary

- **FFmpeg extraction** in `extract_audio()` creates a 16kHz mono MP3 at ~64kbps, reducing video files to approximately 480kB per minute to satisfy upload constraints.
- **Size validation** in `transcribe_video()` triggers `plan_chunks()` and `split_audio()` when audio exceeds 24 MiB, enabling processing of videos of any length.
- **API abstraction** via `_post_whisper()` supports both Groq and OpenAI Whisper endpoints through standardized multipart uploads.
- **Timeline preservation** through `shift_segments()` reassembles chunk-relative timestamps into absolute video timestamps for accurate alignment.
- **Workflow integration** in [`watch.py`](https://github.com/bradautomates/claude-video/blob/main/watch.py) provides a unified interface that orchestrates download, extraction, and transcription.

## Frequently Asked Questions

### Why does Claude Video convert video to MP3 before transcription?

Converting to a **16kHz mono MP3** reduces the data payload by roughly 90% compared to uncompressed video while preserving speech intelligibility. This compression ensures the file stays under the 25 MB upload limits imposed by Whisper API providers, eliminating transfer failures and reducing latency.

### How does the system handle videos longer than 30 minutes?

When `transcribe_video()` detects that the extracted audio exceeds `MAX_UPLOAD_BYTES` (24 MiB), it calculates an optimal chunking strategy using `plan_chunks()`. The audio is then split into smaller segments that fit within API limits, transcribed in parallel or sequence, and reassembled using `shift_segments()` to maintain accurate timestamps.

### Will the transcript timestamps match the original video timeline?

Yes. Although each audio chunk is transcribed independently with timestamps starting at zero, the `shift_segments()` function offsets every segment by the chunk's original position in the video. This produces a continuous transcript where timestamps align precisely with the source video's playback position.

### Which Whisper backends does Claude Video support?

The repository supports both **Groq** and **OpenAI** Whisper endpoints. The `_post_whisper()` function automatically constructs the appropriate multipart request and headers for the selected backend, allowing users to switch providers by changing configuration variables without modifying the transcription logic.