# How timeline_view.py Generates Filmstrip and Waveform Composites for Decision Making

> Discover how timeline_view.py generates filmstrip and waveform composites in video-use for precise editor decision making. Layer images, audio, and text for clear cuts.

- Repository: [Browser Use/video-use](https://github.com/browser-use/video-use)
- Tags: internals
- Published: 2026-07-05

---

**The `render_timeline` function in [`helpers/timeline_view.py`](https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py) composites a single synchronized PNG from a video segment by layering a horizontal filmstrip, an RMS audio waveform ribbon, transcript word labels, and silence markers so editors can make precise visual cut decisions.**

The `browser-use/video-use` toolkit automates video editing workflows, and its [`helpers/timeline_view.py`](https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py) module is the core compositor that translates raw media into a decision-ready visual summary. By locking every graphic element to a single time-to-pixel mapping, the script gives editors a pixel-perfect correlation between frames, audio intensity, spoken words, and pauses. Understanding how [`timeline_view.py`](https://github.com/browser-use/video-use/blob/main/timeline_view.py) generates these filmstrip and waveform composites reveals why it is effective for on-demand drill-down tasks such as verifying retake boundaries or spotting ambiguous silences.

## The `render_timeline` Entry Point

`render_timeline` (lines 84‑130) orchestrates the entire pipeline. It accepts a video path, start/end timestamps, and an optional transcript JSON, then delegates to focused helpers for frame extraction, audio analysis, and text layout before assembling a 1920-pixel-wide canvas.

## Frame Extraction via `extract_frames`

`extract_frames` (lines 37‑62) runs a single `ffmpeg` command per target frame to pull evenly spaced thumbnails between `start` and `end`. The resulting JPEGs are resized to a uniform height of `frame_h = 180` pixels and stored for later horizontal assembly.

## Audio Envelope Generation with `compute_envelope`

`compute_envelope` (lines 68‑111) extracts the segment to a temporary 16 kHz PCM WAV and reads the raw samples with Python’s built-in `wave` module. It computes a windowed **RMS array** of length `samples` (default 2000), normalizes the values to the [0, 1] range, and returns the envelope data that drives the waveform ribbon.

## Transcript Parsing and Silence Detection

`words_in_range` (lines 118‑132) loads the JSON transcript (`{ "words": [...] }`) and returns only entries whose timestamps intersect the `[start, end]` interval. `find_silences` (lines 135‑148) then scans those words for gaps ≥ `threshold` seconds (default 0.4 s), recording the intervals that are later rendered as semi-transparent blue bands.

## Canvas Layout and Coordinate Mapping

Inside `render_timeline`, a blank 1920-pixel canvas is created with `Image.new`. Layout constants define vertical positions for the filmstrip (`filmstrip_y`), waveform (`wave_y`), and label ruler (`label_y`). If the combined width of the thumbnails exceeds available space, the frames are scaled uniformly to fit.

The helper `time_to_x` (lines 159‑162) maps any timestamp in `[start, end]` to an absolute X-pixel coordinate using `(t - start) / (end - start) * strip_span`. All subsequent graphics—silence bands, waveform points, word ticks, and the time ruler—reuse this conversion to maintain perfect temporal alignment.

## Drawing the Filmstrip, Waveform, and Labels

### Pasting the Filmstrip

The resized frames are pasted sequentially onto the canvas beginning at `x = 50`. If scaling was applied, each frame is adjusted and vertically centered. The final span (`strip_span`) is stored for the `time_to_x` conversion.

### Silence Shading

For each silence gap returned by `find_silences`, the code draws a rectangle in the pre-defined `SILENCE` colour—a semi-transparent blue—directly beneath the waveform ribbon.

### Waveform Ribbon Rendering

The normalized envelope array is plotted as two polylines, `points_top` and `points_bot`, tracing the upper and lower halves of the wave. A translucent polygon fills the area between them, producing the familiar ribbon appearance.

### Word-Label Overlays

While iterating over the selected words, the script drops tokens shorter than 50 ms and skips labels that would crowd within 28 pixels of the previous mark. For each accepted word, it draws a short tick on the waveform and renders the text just above it in a 12 pt small font loaded via `load_font` (lines 154‑171).

### Time Ruler and Legend

A ruler with six major ticks is drawn below the waveform, each annotated with an absolute time label formatted as `{t:.2f}s`. If any silences were detected, a legend line is added before the final PNG is written with `canvas.save`.

## CLI and Python Usage Examples

The most common entry point is the command-line interface in [`helpers/timeline_view.py`](https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py).

```bash

# Show a 10-second segment (12 frames) with transcript word labels

python helpers/timeline_view.py \
    path/to/video.mp4  30.0  40.0  \
    --n-frames 12 \
    --transcript path/to/video.json \
    -o /tmp/segment.png

```

For programmatic use, import `render_timeline` and supply the required paths and timestamps.

```python
from pathlib import Path
from helpers.timeline_view import render_timeline

video = Path("movie.mp4")
output = Path("movie_75.5-85.0.png")

render_timeline(
    video=video,
    start=75.5,
    end=85.0,
    out_path=output,
    n_frames=15,
    transcript=Path("movie.json"),
)
print(f"Composite saved to {output}")

```

You can also call `compute_envelope` directly to retrieve the normalized audio envelope as a NumPy array.

```python
import numpy as np
from helpers.timeline_view import compute_envelope
from pathlib import Path

env = compute_envelope(Path("lecture.mp4"), 120.0, 150.0, samples=1000)
print("Peak amplitude:", env.max())

```

## Summary

- `render_timeline` in [`helpers/timeline_view.py`](https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py) orchestrates a three-layer PNG composite: filmstrip, waveform, and transcript labels.
- `extract_frames` (lines 37‑62) pulls uniformly spaced thumbnails via `ffmpeg` and resizes them to 180 px height.
- `compute_envelope` (lines 68‑111) derives a 2000-sample RMS envelope from a temporary 16 kHz WAV for ribbon plotting.
- `words_in_range` (lines 118‑132) and `find_silences` (lines 135‑148) supply the transcript words and silence bands that overlay the waveform.
- A shared `time_to_x` mapping guarantees that frames, audio, words, and pauses align to the same pixel coordinates.

## Frequently Asked Questions

### What dependencies does timeline_view.py require?

The script relies on `ffmpeg` for frame extraction, Pillow for image compositing, NumPy for envelope math, and Python’s built-in `wave` module for PCM audio reading. A transcript JSON is optional but must follow the `{ "words": [...] }` schema when provided.

### Can I change the number of thumbnails or waveform resolution?

Yes. The `n_frames` parameter controls how many thumbnails `extract_frames` requests from `ffmpeg`, and the `samples` parameter passed to `compute_envelope` sets the length of the RMS array that shapes the waveform ribbon.

### Why are some transcript words missing from the composite?

Words shorter than 50 ms are filtered out, and labels are suppressed if they would appear within 28 pixels of the previous label to prevent crowding. This keeps the overlay readable in dense dialogue segments.

### How does the silence detection threshold work?

`find_silences` uses a default gap threshold of 0.4 seconds. Any pause between consecutive transcript words that meets or exceeds this value triggers a semi-transparent blue rectangle under the waveform, making long pauses visually obvious.