Understanding the `takes_packed.md` Format and Phrase-Level Segmentation in Video-Use

The takes_packed.md format is a compact Markdown representation of phrase-level transcripts where each line contains a time range, optional speaker tag, and spoken text, generated by grouping word-level Scribe data based on silence gaps and speaker changes.

The browser-use/video-use repository processes video transcripts to create editable, human-readable formats. The takes_packed.md file serves as the primary phrase-level view of all takes in an edit, offering a token-efficient alternative to raw word-level JSON that is ideal for LLM-driven reasoning or manual cut selection.

What is the takes_packed.md Format?

The takes_packed.md file is a compact, human-readable Markdown document that contains phrase-level transcripts for all takes in a video edit. Each line represents a single spoken phrase prefixed with a millisecond-accurate time range and an optional speaker identifier.

File Structure and Syntax

Each entry follows a strict pattern:

[start-end] S<speaker_id> Phrase text content.

The time stamps use fixed-width formatting with two decimal places (e.g., [0012.34-0014.56]), produced by the format_time function in helpers/pack_transcripts.py (lines 24-27). Speaker tags follow the format S0, S1, etc., indicating which speaker uttered the phrase.

Generation Pipeline

The file is produced by helpers/pack_transcripts.py, which reads raw Scribe JSON word-level transcripts and consolidates them into phrases. The script handles three token types from the Scribe data: "word", "spacing", and "audio_event". After processing, the render_markdown function emits the final Markdown file with duration headers formatted by format_duration (lines 29-35).

How Phrase-Level Segmentation Works

The core segmentation logic resides in the group_into_phrases function within helpers/pack_transcripts.py (lines 38-44). This function transforms word-level data into coherent phrases by detecting natural boundaries in the speech flow.

Input Processing

The function accepts a list of Scribe words entries. Each entry may represent an actual word, a spacing gap between tokens, or an audio event. The algorithm iterates through these tokens to identify where one phrase ends and another begins.

Silence Detection

Silence gaps trigger phrase boundaries when they exceed the configurable threshold. The default silence_threshold is 0.5 seconds. When the function encounters a "spacing" entry indicating a gap longer than this threshold, it flushes the current phrase to the output (lines 88-99).

Speaker Change Detection

Whenever the speaker_id of the next token differs from the current phrase's speaker, the algorithm immediately flushes the current phrase. This ensures that speaker transitions always create distinct phrase boundaries, regardless of timing (lines 107-114).

Flush Logic and Phrase Creation

The flush function (lines 55-82) constructs the final phrase dictionary by:

• Collecting all kept tokens ("word" and "audio_event" entries)
• Concatenating their text content
• Normalizing punctuation
• Recording the phrase's start time, end time, text content, and speaker ID

This phrase object is then appended to the results list, and the accumulator resets for the next phrase.

Working with takes_packed.md Files

Generating Packed Transcripts

To create a takes_packed.md file from your edit directory:

python helpers/pack_transcripts.py --edit-dir path/to/edit

This command reads the raw Scribe JSON files from the specified directory and outputs takes_packed.md containing all grouped phrases.

Parsing the Markdown Programmatically

You can extract structured data from existing packed transcripts using regular expressions:

import re

pattern = re.compile(r'\[(?P<start>\d+\.\d+)-(?P<end>\d+\.\d+)\](?: S(?P<speaker>\d+))? (?P<text>.+)')

with open("takes_packed.md") as f:
    for line in f:
        m = pattern.search(line)
        if m:
            start = float(m["start"])
            end = float(m["end"])
            speaker = int(m["speaker"]) if m["speaker"] else None
            text = m["text"]
            # Process the phrase data...

This pattern captures the start time, end time, optional speaker ID, and text content for each phrase in the file.

Summary

• takes_packed.md is a compact Markdown format containing phrase-level transcripts with time ranges and speaker tags, generated by helpers/pack_transcripts.py.
• Phrase-level segmentation relies on the group_into_phrases function, which groups words based on silence thresholds (default 0.5s) and speaker changes.
• Key implementation files include helpers/pack_transcripts.py for generation logic and utility functions like format_time and format_duration.
• Practical usage involves running the pack script to generate transcripts and using regex patterns to parse them for automated editing workflows.

Frequently Asked Questions

What triggers a new phrase in takes_packed.md generation?

A new phrase is created when either a silence gap exceeds 0.5 seconds or when the speaker ID changes between consecutive words. The group_into_phrases function flushes the current phrase buffer to the output list whenever either condition is met.

Can I adjust the silence threshold for phrase segmentation?

Yes. The silence_threshold parameter in the group_into_phrases function defaults to 0.5 seconds but can be modified when calling the function. Lower values create more phrases with shorter gaps, while higher values merge words into longer phrases.

How does the format handle audio events versus spoken words?

The segmentation logic treats "audio_event" tokens similarly to "word" tokens, including them in the current phrase's text content. Only "spacing" entries trigger boundary detection based on duration thresholds.

Where is the packed transcript used in the video-use workflow?

According to the repository documentation, takes_packed.md serves as the primary reading view for transcripts, referenced in README.md and described in SKILL.md as the lightweight, token-efficient representation ideal for LLM-driven reasoning or manual cut selection during the editing process.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}