# How `timeline_view.py` Composites the Filmstrip, Waveform, and Word Labels

> Discover how timeline_view.py composites filmstrip, waveform and word labels. This Python script creates a multi-layered video visualization by stacking elements onto a single canvas.

- Repository: [Browser Use/video-use](https://github.com/browser-use/video-use)
- Tags: internals
- Published: 2026-07-09

---

**The [`timeline_view.py`](https://github.com/browser-use/video-use/blob/main/timeline_view.py) script composites a multi-layered visualization by vertically stacking an FFmpeg-extracted filmstrip, a filled RMS waveform, and transcript-based word labels and silence shading onto a single PIL canvas.**

The `browser-use/video-use` repository provides a sophisticated visualization utility that transforms video segments into annotated timeline images. In [`helpers/timeline_view.py`](https://github.com/browser-use/video-use/blob/main/helpers/timeline_view.py), the `render_timeline` function orchestrates a six-stage pipeline that combines visual frames, audio envelopes, and transcript data into a unified PNG composite. This compositing process generates clear, information-dense summaries for video analysis by precisely layering distinct visual elements.

## The Visual Layer Architecture

The compositing engine constructs the final image by stacking five distinct visual elements in a specific vertical arrangement, starting with a horizontal offset of 50 pixels and maintaining strategic 20-pixel buffers between major components.

### Filmstrip Frame Extraction

The top visual layer consists of evenly spaced frames extracted via **FFmpeg** using the `extract_frames` function. Each frame is resized to a common height while preserving aspect ratio, then pasted horizontally onto the canvas starting at coordinates `(50, 50)`. The code calculates whether the accumulated width exceeds the canvas boundary, applying uniform scaling when necessary to fit the available space. The resulting strip dimensions are stored in `filmstrip_h` with the vertical position fixed at `filmstrip_y = 50` [source-lines 37-63].

### Waveform Visualization

Directly beneath the filmstrip, the audio layer renders as a filled "mountain" shape representing the RMS envelope. The vertical position is calculated as `wave_y = filmstrip_y + filmstrip_h + 20`, creating a 20-pixel gap between visual elements. The `compute_envelope` function generates a mono 16 kHz PCM segment via FFmpeg, normalizes the amplitude values to the range `[0, 1]`, and maps these to canvas coordinates. The rendering uses `draw.line` and `draw.polygon` with the `WAVE` color constant to create the filled waveform shape [source-lines 68-110].

### Silence Shading and Word Labels

When a transcript is provided, the system enhances the waveform layer with two additional elements. First, `words_in_range` identifies transcript words overlapping the time range, while `find_silences` detects audio gaps exceeding 0.4 seconds. Semi-transparent `SILENCE` rectangles are drawn over these gaps atop the waveform background [source-lines 135-148].

For word labels, each word longer than 50 milliseconds is processed through `time_to_x` to convert temporal data to horizontal pixel coordinates. The code draws a tick mark on the waveform and renders the word text using `small_font` positioned just above the waveform. A collision detection system ensures labels maintain at least 28 pixels of horizontal separation to prevent overlap [source-lines 92-111].

### Time Ruler

Beneath the waveform, a simple time ruler provides temporal context. The code draws six major ticks with absolute timestamp labels using `label_font` and `draw.text`, aligned to the horizontal scale established by the filmstrip and waveform layers [source-lines 114-122].

## The Rendering Pipeline

The `render_timeline` function executes a sequential orchestration to build the final composite:

1. **`extract_frames`** – Pulls N evenly spaced JPGs from the video segment using FFmpeg
2. **Frame Layout** – Resizes frames, computes total width, and determines whether to apply uniform scaling
3. **`compute_envelope`** – Generates normalized RMS envelope data from 16 kHz PCM audio
4. **Transcript Analysis** – Uses `words_in_range` and `find_silences` to identify word positions and silence gaps
5. **Drawing Operations** – Composites filmstrip, waveform background, silence shading, envelope fill, word ticks, labels, and time ruler onto the PIL canvas
6. **PNG Export** – Writes the final image via `canvas.save(out_path, "PNG", optimize=True)` [source-lines 128-132]

## Key Implementation Details

### Coordinate System and Spacing

The canvas employs a fixed horizontal margin of 50 pixels for the filmstrip start. Vertical positioning follows a strict hierarchical offset: the filmstrip anchors at `y = 50`, the waveform sits 20 pixels below the filmstrip bottom, and the time ruler appears beneath the waveform. This consistent spacing ensures visual separation between the filmstrip, waveform, and annotation layers [source-lines 37-63][source-lines 68-110].

### Audio Processing Pipeline

The waveform generation relies on FFmpeg to extract mono 16 kHz PCM audio for the specified time range. The `compute_envelope` function calculates RMS values, normalizes them to a 0-1 range, and maps these to the available vertical space for the polygon fill. The `WAVE` constant from [`helpers/grade.py`](https://github.com/browser-use/video-use/blob/main/helpers/grade.py) defines the specific color used for the filled mountain shape [source-lines 74-89].

### Transcript Integration

Word labels require a JSON transcript typically generated by [`helpers/transcribe.py`](https://github.com/browser-use/video-use/blob/main/helpers/transcribe.py). The `time_to_x` utility function maps temporal data to horizontal pixel positions based on the canvas width and time duration. The rendering engine filters out words shorter than 50 milliseconds and enforces minimum spacing constraints to maintain readability [source-lines 92-111].

## Usage Examples

Generate timeline visualizations using the command-line interface:

```bash

# Basic timeline for seconds 12-18

python helpers/timeline_view.py demo.mp4 12 18 -o demo_12-18.png

```

```bash

# Increase filmstrip density to 20 frames

python helpers/timeline_view.py demo.mp4 12 18 --n-frames 20 -o demo_12-18.png

```

```bash

# Add transcript data for word labels and silence detection

python helpers/timeline_view.py demo.mp4 12 18 --transcript path/to/transcript.json

```

## Summary

- **[`timeline_view.py`](https://github.com/browser-use/video-use/blob/main/timeline_view.py)** composites five distinct visual layers onto a single PIL Image canvas initialized with the `BG` color
- **FFmpeg** powers both frame extraction (`extract_frames`) and audio PCM generation (`compute_envelope`) for the waveform
- **Vertical stacking** follows strict coordinates: filmstrip at `y=50`, waveform 20px below, ruler at bottom
- **Silence detection** overlays semi-transparent `SILENCE` rectangles on audio gaps ≥ 0.4s when transcripts are provided
- **Word labels** include collision detection (28px minimum spacing) and 50ms duration filtering
- **Output** is optimized PNG via `canvas.save()` with the `optimize=True` parameter

## Frequently Asked Questions

### How does [`timeline_view.py`](https://github.com/browser-use/video-use/blob/main/timeline_view.py) handle video frames of different sizes?

The code resizes all extracted frames to a common height while preserving aspect ratios. It calculates the total width required for the filmstrip and applies uniform scaling if the width exceeds the canvas boundary, ensuring the entire strip fits within the available horizontal space [source-lines 37-63].

### What audio format does the waveform visualization use?

The tool extracts mono 16 kHz PCM audio using FFmpeg, then computes the RMS envelope to create the filled mountain shape. This normalized envelope provides the y-coordinate data for the `draw.polygon` operations that render the waveform visualization [source-lines 74-89].

### Can I customize the colors used in the timeline visualization?

Yes, the script references color constants including `WAVE`, `SILENCE`, and `BG` imported from [`helpers/grade.py`](https://github.com/browser-use/video-use/blob/main/helpers/grade.py). Modifying these constants changes the waveform fill color, silence overlay shading, and canvas background color respectively.

### How does the script prevent word labels from overlapping?

The rendering engine enforces a minimum 28-pixel spacing between adjacent word labels during the drawing phase. Additionally, it filters out words shorter than 50 milliseconds, ensuring only significant utterances receive annotations and maintaining visual clarity [source-lines 92-111].