How Claude Video Parses VTT Captions and Filters Them by Time Range

Claude Video parses YouTube VTT subtitle files using parse_vtt() to extract timestamped segments, then filters them with filter_range() based on optional --start and --end CLI arguments, falling back to Whisper transcription when native captions are unavailable.

This article explains how the bradautomates/claude-video repository handles WebVTT (.vtt) caption processing. Understanding these internals helps you customize transcript extraction, debug timing issues, or extend the filtering logic for your own video analysis pipelines.

Parsing VTT Captions with parse_vtt

The parse_vtt function in skills/watch/scripts/transcribe.py converts raw VTT files into structured segment dictionaries. This is the first stage of caption processing.

Core Parsing Logic

The function processes VTT files line-by-line using two regular expressions:

• TS_RE – matches timestamp lines in HH:MM:SS.ms --> HH:MM:SS.ms format
• TAG_RE – strips HTML-style tags like <c> or <00:00:01.000>

Here's the implementation flow from lines 24–52:

from transcribe import parse_vtt

segments = parse_vtt("path/to/subtitles.vtt")

# Result: [{'start': 0.0, 'end': 2.33, 'text': 'Hello world'}, ...]

The parser performs four key operations:

1. Splits the file into lines and iterates with state tracking
2. Extracts timestamps when TS_RE matches, converting to float seconds
3. Collects cue text until a blank line, applying TAG_RE to remove markup
4. Rounds start/end times and appends to the segment list

Deduplicating Rolling Cues with _dedupe

VTT files often contain consecutive identical captions or cues that extend previous text. The _dedupe helper (lines 55–67) collapses these redundancies:

• Merges consecutive segments with identical text
• Combines captions where one extends another (rolling subtitles)
• Returns the cleaned, compacted segment list

This deduplication is critical for clean transcripts, especially with YouTube's automatically generated captions that frequently repeat phrases across overlapping time windows.

Filtering Segments by Time Range

The filter_range function (lines 70–80) implements the second stage: temporal filtering. It accepts parsed segments plus optional start_seconds and end_seconds parameters.

Overlap-Based Filtering Logic

The function uses inclusive overlap detection rather than strict containment:

from transcribe import filter_range

# Extract only segments overlapping the 10-40 second window

windowed = filter_range(segments, start_seconds=10.0, end_seconds=40.0)

A segment is kept when:

• segment["end"] >= lo (segment ends after range start)
• segment["start"] <= hi (segment starts before range end)

This overlap-based approach ensures partial segments aren't clipped, preserving context at range boundaries.

Edge Case Handling

• No range specified – returns the original list unmodified
• Empty result – returns [] when no segments overlap the range
• Millisecond precision – times are stored as floats for sub-second accuracy

CLI Integration in watch.py

The watch entry point orchestrates parsing and filtering based on user arguments.

Time Argument Parsing

User-friendly time strings (00:01:15) are converted to seconds via parse_time:

/watch https://youtu.be/xyz --start 00:01:15 --end 00:02:00

The CLI then invokes the filtering pipeline (lines 63–66):

# Parsed from --start/--end arguments

start_sec = parse_time(args.start)  # e.g., 75.0

end_sec = parse_time(args.end)      # e.g., 120.0

transcript_segments = filter_range(transcript_segments, start_sec, end_sec)

Whisper Fallback Compatibility

When VTT parsing fails or no subtitles exist, transcribe_video (lines 39–50) generates segments via OpenAI Whisper. These segments follow the identical {"start", "end", "text"} schema, making them directly compatible with the same filter_range call:

# Fallback path in watch.py

else:
    print("No subtitles found, transcribing with Whisper...")
    transcript_segments = transcribe_video(dl["video_path"])
    # Same filtering applies regardless of source

    transcript_segments = filter_range(transcript_segments, start_sec, end_sec)

Source File Reference

File	Lines	Responsibility
skills/watch/scripts/transcribe.py	24–52	parse_vtt() – VTT parsing and tag stripping
skills/watch/scripts/transcribe.py	55–67	_dedupe() – rolling cue deduplication
skills/watch/scripts/transcribe.py	70–80	filter_range() – temporal filtering
skills/watch/scripts/watch.py	39–50	Whisper fallback handling
skills/watch/scripts/watch.py	63–66	CLI argument processing and filter invocation

Summary

• parse_vtt converts raw VTT files to structured segments with cleaned text and float timestamps
• _dedupe collapses redundant consecutive captions for readable transcripts
• filter_range applies overlap-based filtering that preserves partial segments at range boundaries
• The CLI bridge in watch.py coordinates these functions, with identical filtering logic applied to both native VTT and Whisper fallback transcripts
• All components use a consistent segment schema enabling interchangeable data sources

Frequently Asked Questions

How does Claude Video handle malformed VTT files?

When VTT parsing fails or no subtitle file exists, the system falls back to Whisper transcription via transcribe_video() in watch.py (lines 39–50). The resulting segments use the same dictionary structure, so subsequent filter_range calls work identically regardless of the source.

Why use overlap-based filtering instead of strict containment?

The filter_range function keeps any segment where seg["end"] >= lo AND seg["start"] <= hi. This prevents clipping mid-sentence captions that partially extend into the requested range, ensuring transcript readability and context preservation at time boundaries.

Can I parse VTT files without using the full watch command?

Yes. Import parse_vtt directly from transcribe.py to process VTT files in isolation. The function has no external dependencies beyond standard library modules, making it portable for other transcription workflows.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}