How the Packed Transcript Format Enables LLM-Based Video Editing

The packed transcript format compresses raw word-level JSON into phrase-level markdown with precise timestamps, reducing token count by approximately 10× while providing LLMs with structured time ranges to generate precise video edit decisions.

The browser-use/video-use repository introduces a specialized pipeline that transforms noisy speech-to-text output into a machine-readable format optimized for automated editing. The packed transcript format serves as the critical bridge between raw transcription data and Large Language Model (LLM) reasoning, enabling AI-driven video editing workflows that require frame-accurate timing information.

Understanding the Packed Transcript Pipeline

The conversion process in helpers/pack_transcripts.py implements three distinct transformation stages to prepare transcript data for LLM consumption.

Phrase Grouping via Silence Detection

The group_into_phrases function processes the raw word list from Scribe JSON transcripts, aggregating individual words into logical phrases based on two triggers: a silence duration exceeding the silence_threshold (default 0.5 seconds) or a detected speaker change. This logic appears in lines 38-44 of the implementation.

Each resulting phrase object contains:

• start and end timestamps
• Concatenated text content
• Optional speaker_id for multi-speaker content

This grouping reduces hundreds of individual word objects into a modest list of semantic phrases, dramatically lowering the token payload sent to the LLM.

Human-Readable Timestamp Formatting

Helper functions format_time and format_duration (lines 24-36) convert raw float timestamps into fixed-width strings. The format uses six-digit seconds with two decimal places (e.g., 0012.34), ensuring consistent alignment in the markdown output and unambiguous parsing by both humans and language models.

Markdown Rendering with Structured Metadata

The render_markdown function (lines 45-62) emits the final takes_packed.md file, which includes:

• A header explaining the grouping rules
• Duration statistics
• Phrase-level entries prefixed by [start-end] ranges and optional speaker tags (e.g., S1)

By design, the output places each phrase on its own line with predictable delimiters, creating a plain-text structure that LLMs can parse using simple regex patterns or few-shot examples.

Why LLMs Process Packed Transcripts Efficiently

Raw transcription JSON contains excessive metadata—word-level confidence scores, individual word timings, and punctuation tokens—that consumes valuable context window space. The packed transcript format eliminates this noise while preserving the precise timing data required for video editing.

The format enables three key LLM capabilities:

• Temporal reasoning: Fixed-width [start-end] markers allow the model to map textual references directly to video timecodes without ambiguity.
• Syntactic clarity: Each phrase represents a complete syntactic unit, providing the LLM with contextual boundaries for making edit decisions.
• Token efficiency: As implemented in lines 99-102, the script achieves approximately a 10× reduction in token count compared to the raw JSON input, allowing longer transcripts to fit within model context windows.

Generating Edit Decisions with LLMs

The structured output in takes_packed.md enables LLMs to generate Edit Decision Lists (EDLs) by selecting, reordering, or trimming specific time ranges. The predictable markdown syntax allows programmatic extraction of timing data.

Here is a Python implementation that consumes the packed transcript and queries an LLM for editing suggestions:

import pathlib
import re
from openai import OpenAI

# Read the packed transcript

md = pathlib.Path("my_edit/takes_packed.md").read_text()

# Extract all phrase lines with regex

pattern = r'^\s*\[(\d{6}\.\d{2})-(\d{6}\.\d{2})\](?: S(\d+))? (.+)$'
phrases = re.findall(pattern, md, re.MULTILINE)

# Construct the prompt

prompt = "Given the following video transcript fragments, suggest three cuts to shorten the interview:\n"
for start, end, speaker, text in phrases[:20]:
    prompt += f"[{start}-{end}] (Speaker {speaker or 'unknown'}): {text}\n"

# Query the LLM

client = OpenAI()
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": prompt}]
)
print(response.choices[0].message.content)

The LLM responds with edit instructions referencing the exact [start-end] ranges, which can be converted directly into cutting commands or EDL formats.

Running the Transcript Packer

To generate the packed transcript from raw Scribe JSON files, execute the following command from the repository root:

python helpers/pack_transcripts.py --edit-dir my_edit

This script reads all JSON files in my_edit/transcripts/, applies the phrase grouping logic, and writes my_edit/takes_packed.md. The resulting file serves as the primary input for LLM-based editing workflows, as detailed in the repository's README.md.

Summary

• The packed transcript format converts raw word-level JSON into phrase-level markdown, achieving approximately 10× token reduction for LLM processing.
• The group_into_phrases function in helpers/pack_transcripts.py creates semantic boundaries using a 0.5-second silence threshold and speaker changes.
• Fixed-width timestamp formats ([0000.00-0003.25]) enable precise temporal mapping between text content and video timecodes.
• The resulting takes_packed.md file provides a structured, plain-text interface that LLMs can parse to generate edit decisions, cuts, and rearrangements.

Frequently Asked Questions

What is the difference between raw transcripts and packed transcripts?

Raw transcripts contain word-level JSON objects with individual timings and confidence scores, creating excessive token overhead for LLMs. Packed transcripts aggregate these into phrase-level markdown entries with consolidated timestamps, preserving only the timing and content data necessary for video editing decisions.

How does the silence threshold affect phrase grouping?

The silence_threshold parameter (default 0.5 seconds) in group_into_phrases determines when the packer splits words into separate phrases. Lower values create more granular phrases, while higher values merge words across longer pauses. This threshold directly impacts the granularity of edit points available to the LLM.

Can packed transcripts handle multiple speakers?

Yes. The format detects speaker changes during the grouping phase and appends speaker tags (e.g., S1, S2) to the respective phrase lines. This allows LLMs to perform speaker-specific edits, such as removing filler words from one participant while preserving another's content.

How do I integrate packed transcripts with my LLM workflow?

Read the takes_packed.md file generated by helpers/pack_transcripts.py, extract timestamp ranges using regex patterns, and include them in your LLM prompt with clear instructions about the video editing task. The structured format requires no specialized parsing libraries—standard string operations suffice for preparing the context.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}