How Chunk Offsets Are Handled When Transcribing Audio in Multiple Parts in Claude-Video

Chunk offsets in the claude-video repository are managed by calculating time-based boundaries for each audio slice, then adjusting Whisper API segment timestamps by adding the original offset to align the final transcript with the original video timeline.

When processing large video files with the /watch skill, the bradautomates/claude-video repository splits audio data into smaller payloads to respect API limits. The system ensures that transcribed segments reflect their true positions in the original media by mathematically shifting timestamps during the re-assembly phase. This approach allows the Whisper API to process audio in isolation while maintaining chronological accuracy across the entire transcript.

Planning Chunk Boundaries Based on Payload Size

The transcription pipeline begins by determining how to divide the audio file without exceeding the MAX_UPLOAD_BYTES limit. The plan_chunks function in skills/watch/scripts/whisper.py serves as the entry point for this calculation.

The plan_chunks Function

Located at lines 45-62, plan_chunks computes the necessary segmentation by accepting the total audio duration and file size. It returns a list of tuples containing (offset, duration) pairs, where each offset represents the start time in seconds for that specific chunk relative to the original video timeline.

# Example: split a 300-second audio file (~30 MiB) into chunks < 24 MiB

chunks = plan_chunks(total_seconds=300, total_bytes=30_000_000)

# → [(0.0, 120.0), (120.0, 120.0), (240.0, 60.0)]

This planning stage is critical because it establishes the temporal anchor points that will later be used to reconstruct accurate timestamps. Each tuple represents a discrete time window that Whisper will process independently, starting from time zero within that slice.

Adjusting Timestamps Across Audio Segments

Since Whisper returns timestamps relative to the start of each individual chunk (always beginning at 0.0 seconds), the system must apply a temporal shift to synchronize these segments with the original video. This adjustment happens in two coordinated steps within skills/watch/scripts/whisper.py.

The shift_segments Function

The shift_segments function (lines 32-45) performs the mathematical correction by creating new segment dictionaries with adjusted start and end values. It accepts the Whisper response segments and an offset_seconds parameter, returning a list where each timestamp has been incremented by the chunk's original offset.

def shift_segments(segments: list[dict], offset_seconds: float) -> list[dict]:
    # Creates new segments with start/end shifted by offset_seconds

    return [
        {
            "start": seg["start"] + offset_seconds,
            "end": seg["end"] + offset_seconds,
            "text": seg["text"]
        }
        for seg in segments
    ]

This function ensures that a segment ending at 30.0 seconds in the second chunk (which starts at 120.0 seconds in the original video) is recorded as ending at 150.0 seconds in the final transcript.

The transcribe_chunks Orchestrator

The transcribe_chunks function (lines 71-80) manages the iteration and re-assembly process. It accepts the list of chunk tuples generated by plan_chunks and a transcribe_one callback function that handles the actual Whisper API communication.

For each chunk, the orchestrator:

1. Extracts the offset and duration from the planning tuple
2. Calls transcribe_one to retrieve the raw segments
3. Applies shift_segments using the chunk's offset value
4. Concatenates the corrected segments into the master transcript

Failed chunks are logged and skipped without interrupting the pipeline, but successful chunks retain their offset-corrected timestamps to preserve timeline continuity.

Complete Workflow Implementation

The following implementation demonstrates how these three functions coordinate to handle chunk offsets when transcribing audio in multiple parts:

from pathlib import Path
from skills.watch.scripts.whisper import plan_chunks, shift_segments

def transcribe_one(path: Path) -> list[dict]:
    """
    Calls Whisper API for a single chunk.
    Returns segments with timestamps starting at 0.
    """
    # ... API call implementation ...

    return [{"start": 0.0, "end": 5.2, "text": "Example content"}]

def transcribe_chunks(chunks, transcribe_func):
    """
    Orchestrates multi-part transcription with offset correction.
    """
    full_transcript = []
    
    for offset, duration in chunks:
        # Transcribe individual chunk (timestamps start at 0)

        segments = transcribe_func(Path(f"chunk_{offset}.mp3"))
        
        # Shift timestamps by the chunk's original offset

        corrected_segments = shift_segments(segments, offset)
        full_transcript.extend(corrected_segments)
    
    return full_transcript

# Execution pipeline

audio_duration = 300  # seconds

file_size = 30_000_000  # bytes (~30 MiB)

# 1. Plan chunks based on MAX_UPLOAD_BYTES

chunks = plan_chunks(audio_duration, file_size)

# 2. Transcribe and automatically adjust offsets

final_transcript = transcribe_chunks(chunks, transcribe_one)

# All segment timestamps now reference the original 300-second timeline

According to the bradautomates/claude-video source code, this architecture ensures that the final transcript accurately reflects when words were spoken in the original media, regardless of how the audio was divided for API processing.

Summary

• Chunk planning uses plan_chunks in skills/watch/scripts/whisper.py (lines 45-62) to generate (offset, duration) tuples based on MAX_UPLOAD_BYTES constraints.
• Timestamp correction is performed by shift_segments (lines 32-45), which adds the chunk's offset to each segment's start and end times.
• Re-assembly is managed by transcribe_chunks (lines 71-80), which iterates through planned chunks, applies the offset correction, and concatenates results while logging failed segments.
• The system ensures chronological accuracy by treating Whisper's relative timestamps as deltas to be adjusted by the pre-calculated offset values.

Frequently Asked Questions

How does the system handle chunks that fail to transcribe?

Failed chunks are logged and skipped during the transcribe_chunks iteration, but the pipeline continues processing remaining chunks. Successful chunks retain their offset-corrected timestamps, meaning gaps may exist in the final transcript where failed segments occurred, but the surrounding timestamps remain accurate relative to the original video.

What determines the size of each audio chunk?

The plan_chunks function calculates chunk boundaries using the MAX_UPLOAD_BYTES constant, which represents the maximum payload size for the Whisper API. The function divides the total file size by this limit to determine how many segments are needed, then distributes the total duration evenly across those segments to create the (offset, duration) tuples.

Why do timestamps need to be shifted after transcription?

Whisper API returns timestamps relative to the beginning of each audio chunk (always starting at 0.0 seconds). Without the shift_segments adjustment implemented in skills/watch/scripts/whisper.py, the second chunk would appear to start at 0.0 seconds instead of its actual position (e.g., 120.0 seconds) in the original video timeline.

Can the chunking strategy handle variable bitrate audio files?

Yes, because plan_chunks accepts both total_seconds and total_bytes as parameters. It calculates the byte-to-second ratio to determine appropriate durations that keep each chunk under MAX_UPLOAD_BYTES, making the offset calculation accurate regardless of the audio encoding or bitrate.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}