# Understanding the Timeline Visualization Generated by `timeline_view.py` in video-use

> Explore the timeline visualization from video-use. This Python script generates a PNG layering filmstrip frames, audio waveform, transcript, and metadata for video analysis.

- Repository: [Browser Use/video-use](https://github.com/browser-use/video-use)
- Tags: deep-dive
- Published: 2026-08-06

---

**[`timeline_view.py`](https://github.com/browser-use/video-use/blob/main/timeline_view.py) produces a single PNG image that layers filmstrip frames, audio waveform, transcript overlays, and time-aligned metadata to give developers a visual drill-down of any video segment.**

The [`timeline_view.py`](https://github.com/browser-use/video-use/blob/main/timeline_view.py) module in the browser-use/video-use repository is a lightweight visualization tool for audio-video analysis. It generates **on-demand timeline PNGs** that combine multiple data streams into one cohesive image, making it ideal for quality control, editing workflows, and research annotation. This guide breaks down every visual layer, implementation detail, and usage pattern.

## Core Architecture: Five Visual Layers

The timeline visualization is constructed as a composite of five distinct layers, each with dedicated rendering logic in [`helpers/timeline_view.py`](https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py).

### Filmstrip Frame Extraction

The topmost layer displays **evenly spaced video frames** from the specified `[start, end]` range.

- Frame extraction is handled by `extract_frames` (lines 37-62)
- Frames are resized to a **uniform height of 180 px**
- Layout logic in `render_timeline` (lines 99-154) determines whether frames render at full size or scale uniformly to fit canvas width

The number of frames is user-configurable via `--n-frames` (default varies by segment duration).

### Waveform Ribbon

A **mono-RMS audio envelope** appears directly beneath the filmstrip as a filled blue polygon.

- Color constant: `WAVE = (140, 180, 255)` — a medium-brightness blue
- Audio is extracted to temporary 16 kHz PCM WAV, then windowed and normalized
- Computation happens in `compute_envelope` (lines 68-110)
- Final rendering occurs in lines 175-291 as a filled polygon representing amplitude over time

This gives immediate visual feedback on speech presence, music, and silence patterns.

### Transcript Overlays

When a transcript JSON is supplied, the visualization adds **word-level labels** and **silence highlighting**:

| Element | Threshold | Visual Treatment |
|---------|-----------|----------------|
| Word labels | ≥ 50 ms duration | Text rendered above waveform at time position |
| Silence bands | ≥ 400 ms gaps | Semi-transparent blue shading: `SILENCE = (50, 80, 120, 120)` |

- Word filtering: `words_in_range` (lines 118-148)
- Silence detection: `find_silences` (within same range)
- Drawing implementation: lines 267-326

Words position proportionally along the timeline, creating visual alignment between audio, transcript, and frames.

### Ruler and Metadata Labels

Structural elements anchor the visualization:

- **Header line**: Video filename, time range (`start-end`), and frame count
- **Time ruler**: Six major ticks below the waveform with proportional time labels
- Header rendering: lines 225-244
- Ruler and ticks: lines 313-322

### Color System and Typography

The dark-theme palette is defined as constants (lines 76-81):

```python
BG = (18, 18, 22)      # Near-black background

FG = (235, 235, 235)   # Off-white primary text

DIM = (110, 110, 120)  # Muted gray for secondary elements

```

Font loading via `load_font` (lines 154-170) attempts common system monospaced fonts (Menlo, Monaco, Consolas, DejaVu Sans Mono) with a default PIL fallback.

## Output Specifications

The generated timeline visualization has **fixed dimensions**:

- Default canvas: approximately **1920 × 540 pixels**
- Single PNG output (no SVG or interactive formats)
- Designed for **single-segment inspection**, not batch per-utterance processing

This constraint keeps rendering fast and file sizes manageable for documentation and review workflows.

## Command-Line Usage

Generate a basic timeline with frame extraction only:

```bash
python helpers/timeline_view.py \
    path/to/video.mp4 30.0 42.0 \
    --n-frames 12 \
    -o output/timeline.png

```

Add transcript overlays for word-level annotation:

```bash
python helpers/timeline_view.py \
    path/to/video.mp4 30.0 42.0 \
    --transcript path/to/transcript.json \
    --n-frames 10

```

## Python API Integration

Embed the visualization pipeline directly in Python scripts:

```python
from pathlib import Path
from helpers.timeline_view import render_timeline

video = Path("video.mp4")
start, end = 30.0, 42.0
out = Path("timeline.png")

render_timeline(
    video=video,
    start=start,
    end=end,
    out_path=out,
    n_frames=12,
    transcript=Path("transcript.json"),  # optional, adds overlays

)

```

The function signature accepts `Path` objects for all file parameters, with `transcript` defaulting to `None` for waveform-only output.

## Related Source Files

| File | Purpose |
|------|---------|
| [`helpers/timeline_view.py`](https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py) | Core implementation: frame extraction, envelope computation, transcript parsing, PNG rendering |
| [`helpers/transcribe.py`](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py) | Transcript generation utility producing JSON files consumed by timeline view |
| [`helpers/render.py`](https://github.com/browser-use/video-use/blob/main/helpers/render.py) | General image-rendering utilities for extension development |
| [`helpers/grade.py`](https://github.com/browser-use/video-use/blob/main/helpers/grade.py) | Grading utilities that may reference timeline images for visual inspection |

## Summary

- **[`timeline_view.py`](https://github.com/browser-use/video-use/blob/main/timeline_view.py) generates fixed-size PNG visualizations** (~1920×540px) combining filmstrip frames, RMS waveform, transcript words, and silence shading
- **Five composited layers**: frame filmstrip, blue waveform ribbon, word overlays, silence bands, and time ruler with metadata header
- **Configurable via CLI or Python API**: frame count, time range, and optional transcript JSON
- **Dark-themed design** with system font fallback and accessibility-conscious contrast ratios
- **On-demand, not streaming**: Optimized for segment inspection rather than real-time or batch processing

## Frequently Asked Questions

### What audio format does the waveform ribbon use?

The waveform ribbon is computed from a **temporary 16 kHz mono PCM WAV** extracted from the source video. The RMS envelope is windowed, normalized, and rendered as a filled polygon in `compute_envelope` (lines 68-110).

### Can I customize the color scheme?

The color constants (`BG`, `FG`, `DIM`, `WAVE`, `SILENCE`) are hardcoded at lines 76-81 in [`timeline_view.py`](https://github.com/browser-use/video-use/blob/main/timeline_view.py). Modification requires editing the source code directly; there are no runtime color parameters.

### Why is there a 50 ms threshold for word labels?

Words shorter than 50 ms are filtered out in `words_in_range` (lines 118-148) to prevent visual clutter from brief artifacts, breath sounds, or transcription noise. This threshold balances detail density with readability.

### How does silence detection work?

Silence bands are identified by gaps between words in the transcript JSON that exceed **400 milliseconds**. These regions receive semi-transparent blue shading (`SILENCE = (50, 80, 120, 120)`) to visually distinguish inactive audio segments, implemented in `find_silences` and rendered in lines 267-326.