# Data Flow from transcribe.py to pack_transcripts.py to takes_packed.md for LLM Transcript Reading

> Understand the data flow from transcribe.py to pack_transcripts.py to takes_packed.md. Learn how video transcripts are processed for LLM reading easily.

- Repository: [Browser Use/video-use](https://github.com/browser-use/video-use)
- Tags: internals
- Published: 2026-07-04

---

**The pipeline extracts audio from video files into word-level JSON transcripts using [`transcribe.py`](https://github.com/browser-use/video-use/blob/main/transcribe.py), aggregates those words into phrase-level segments via [`pack_transcripts.py`](https://github.com/browser-use/video-use/blob/main/pack_transcripts.py) based on silence thresholds and speaker changes, and outputs a compact [`takes_packed.md`](https://github.com/browser-use/video-use/blob/main/takes_packed.md) file that serves as the primary reading material for LLMs.**

The browser-use/video-use repository implements a three-stage pipeline that transforms raw video into a structured, LLM-optimized markdown format. This data flow from [`transcribe.py`](https://github.com/browser-use/video-use/blob/main/transcribe.py) to [`pack_transcripts.py`](https://github.com/browser-use/video-use/blob/main/pack_transcripts.py) to [`takes_packed.md`](https://github.com/browser-use/video-use/blob/main/takes_packed.md) ensures that language models receive high-resolution temporal cues while minimizing token count by grouping words into meaningful phrases.

## Stage 1: Audio Extraction and JSON Transcription

The process begins in [`helpers/transcribe.py`](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py), which handles audio preprocessing and API communication with ElevenLabs Scribe.

### Extracting Mono 16 kHz Audio

The script calls `extract_audio` to generate a mono 16 kHz WAV file from the source video, optimizing the format for speech recognition APIs.

### Caching Word-Level JSON Output

After uploading to ElevenLabs Scribe, the API returns a JSON payload containing word-level timestamps, speaker IDs, and audio events. According to the source code at lines 23-27, this response is written to `<edit_dir>/transcripts/<video-stem>.json` and cached for subsequent runs, preventing redundant API calls.

## Stage 2: Phrase-Level Aggregation

The aggregation logic resides in [`helpers/pack_transcripts.py`](https://github.com/browser-use/video-use/blob/main/helpers/pack_transcripts.py), which transforms granular word data into coherent phrases suitable for LLM consumption.

### Scanning Transcript Directories

The packer initializes by scanning the transcripts directory. As shown at lines 87-89, the code uses `json_files = sorted(transcripts_dir.glob("*.json"))` to collect all cached transcription files for processing.

### Grouping Words into Phrases

The core logic relies on the `group_into_phrases` function, which iterates through the `"words"` array from each JSON file. The function implements a `flush` mechanism (lines 55-81) that aggregates words into phrase dictionaries containing `start`, `end`, `text`, and `speaker_id` fields.

### Handling Speaker Changes and Silence Thresholds

Phrases are flushed based on two specific conditions documented in the source:

- **Silence detection**: When a spacing entry (`type == "spacing"`) exceeds the default threshold of 0.5 seconds (lines 90-98)
- **Speaker transitions**: When the `speaker_id` changes between consecutive words (lines 107-110)

## Stage 3: Markdown Generation for LLM Consumption

The final output stage converts the structured phrase data into a human-readable, machine-parseable format.

### Time-Annotated Format

The `render_markdown` function generates lines using the `format_time` utility to create precise time ranges. Each line follows the pattern documented at lines 44-61:

```markdown
[002.52-005.36] S0 Ninety percent of what a web agent does is completely wasted.

```

The optional speaker tag (`S0`, `S1`, etc.) prefixes the text content when speaker identification is available.

### The LLM Reading View

As documented in [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) (lines 12-14), the resulting [`takes_packed.md`](https://github.com/browser-use/video-use/blob/main/takes_packed.md) file serves as the LLM's primary reading view. The file is written to `<edit_dir>/takes_packed.md` (lines 94-95 in [`pack_transcripts.py`](https://github.com/browser-use/video-use/blob/main/pack_transcripts.py)), providing a compact representation that preserves temporal context while reducing token overhead compared to raw word-level data.

## Running the Pipeline

Execute the scripts sequentially to process video content.

Transcribe a single video to create the JSON cache:

```bash
python helpers/transcribe.py path/to/video.mp4 --edit-dir ./my_edit

```

Pack all transcripts into the LLM-readable markdown:

```bash
python helpers/pack_transcripts.py --edit-dir ./my_edit

```

The generated [`takes_packed.md`](https://github.com/browser-use/video-use/blob/main/takes_packed.md) organizes content by video segments with phrase-level granularity:

```markdown

## C0103  (duration: 43.0s, 8 phrases)

  [002.52-005.36] S0 Ninety percent of what a web agent does is completely wasted.
  [006.08-006.74] S0 We fixed this.

```

## Summary

- **[`helpers/transcribe.py`](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py)** extracts mono 16 kHz audio and caches word-level JSON transcripts from ElevenLabs Scribe to `<edit_dir>/transcripts/<video-stem>.json`
- **[`helpers/pack_transcripts.py`](https://github.com/browser-use/video-use/blob/main/helpers/pack_transcripts.py)** scans all JSON files, uses `group_into_phrases` to aggregate words based on 0.5-second silence thresholds and speaker changes, and stores phrase data with precise timestamps
- **[`takes_packed.md`](https://github.com/browser-use/video-use/blob/main/takes_packed.md)** is generated as the final artifact, formatted with time ranges and speaker IDs, serving as the optimized reading source for LLMs according to the repository's [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) documentation

## Frequently Asked Questions

### What triggers a new phrase boundary in the packing algorithm?

The `group_into_phrases` function in [`pack_transcripts.py`](https://github.com/browser-use/video-use/blob/main/pack_transcripts.py) flushes the current phrase when either a silence entry exceeds 0.5 seconds or when the `speaker_id` changes between words, ensuring phrases align with natural speech patterns and speaker turns.

### Why does the pipeline use a two-step process instead of direct markdown generation?

The intermediate JSON caching in [`transcribe.py`](https://github.com/browser-use/video-use/blob/main/transcribe.py) allows for incremental processing and API cost optimization, while the separate packing stage enables algorithmic refinement of phrase boundaries without re-querying the transcription API.

### How does the markdown format preserve temporal information for LLMs?

Each line in [`takes_packed.md`](https://github.com/browser-use/video-use/blob/main/takes_packed.md) includes a formatted time range (e.g., `[002.52-005.36]`) generated by the `format_time` function, allowing LLMs to reference specific moments in the video while reading compact phrase-level text rather than individual words.

### Where is the final markdown file located and how is it referenced?

The file is written to `<edit_dir>/takes_packed.md` by default, as implemented at lines 94-95 in [`pack_transcripts.py`](https://github.com/browser-use/video-use/blob/main/pack_transcripts.py), and is explicitly documented in [`SKILL.md`](https://github.com/browser-use/video-use/blob/main/SKILL.md) as the primary source document for LLM-driven transcript reading tasks.