How ElevenLabs Scribe Provides Word-Level Timestamps for Precise Video Editing

ElevenLabs Scribe generates word-level timestamps by setting the timestamps_granularity parameter to "word" in the API request, returning a JSON response where each entry in the words array contains the spoken text alongside precise start and end times in seconds.

The browser-use/video-use repository leverages ElevenLabs Scribe to convert video audio into granular transcripts with per-word timing data. This capability enables frame-accurate editing, automated subtitle generation, and speaker-aware timeline visualization. The implementation centers on helpers/transcribe.py, which orchestrates audio extraction, API communication, and downstream consumption of timestamp data.

How ElevenLabs Scribe Generates Word-Level Timestamps

Audio Preparation and Extraction

Before sending data to the Scribe API, the video audio must meet specific format requirements. In helpers/transcribe.py, the extract_audio() function processes the input video using ffmpeg to produce a mono 16kHz WAV file. This standardized format ensures optimal speech recognition accuracy and consistent processing latency. The extraction occurs at lines 49-55, creating the necessary audio foundation for precise timestamp generation.

API Configuration for Granular Timing

The core timestamp functionality is activated through a specific payload configuration in the call_scribe() function. When POSTing to https://api.elevenlabs.io/v1/speech-to-text, the request body deliberately sets "timestamps_granularity": "word" at lines 66-70. This parameter instructs Scribe to return boundary timestamps for every individual spoken token rather than sentence or paragraph-level timings.

The request also enables additional metadata by setting "diarize": "true" for speaker identification and "tag_audio_events": "true" for non-speech audio detection. These flags operate alongside the word-level granularity to provide a comprehensive annotation of the audio timeline.

Response Structure and Data Format

Upon successful processing, the API returns a JSON object containing a top-level words array. Each element in this array is an object with three essential properties: the spoken word as text, a start time in seconds, and an end time in seconds. This structure allows downstream components to calculate exact durations and positions for every token in the transcript.

Implementation in the Video-Use Repository

The Transcription Orchestration Script

The helpers/transcribe.py script serves as the central hub for the video-use transcription workflow. It manages the entire pipeline from raw video input to structured timestamp data, handling file conversion, API authentication, and error retry logic. The script expects an ElevenLabs API key configured according to the repository's install.md documentation.

Users invoke this functionality through the command line by specifying the video path, output directory, language, and speaker count. The script automatically manages the upload process and stores the resulting JSON transcript for subsequent processing.

Caching for Deterministic Timestamps

To ensure consistency across editing sessions and avoid unnecessary API costs, the repository implements intelligent caching at lines 100-108 of helpers/transcribe.py. If a transcript already exists for a given video, the script skips re-uploading and re-processing, returning the cached word-level timestamp data instead. This deterministic approach guarantees that repeated runs of the editing pipeline produce identical timing results, crucial for version control and collaborative workflows.

Downstream Applications of Word-Level Data

Interactive Timeline Visualization

The helpers/timeline_view.py component consumes the words array to generate interactive SVG timelines. At lines 123-132, the script iterates through each word's start and end timestamps to position visual elements on a temporal axis. This visualization allows editors to identify precise cut points, navigate to specific spoken phrases, and understand the temporal relationship between speakers.

Frame-Accurate Subtitle Rendering

For subtitle generation, helpers/render.py aligns video frames with word timings to create precise text overlays. The rendering logic at lines 302-311 uses the per-word timestamps to determine when each subtitle should appear and disappear on screen. This frame-accurate alignment ensures that subtitles sync perfectly with speech, avoiding the lag or lead common with less granular timestamp systems.

Transcript Packaging and Review

The helpers/pack_transcripts.py script aggregates the words list into compact, human-readable markdown formats at lines 128-138. By preserving the temporal data in the output, editors can review transcripts with inline timing references, making it easier to identify sections requiring edits without returning to the video player.

Practical Code Examples

To transcribe a video with word-level timestamps:

python helpers/transcribe.py path/to/video.mp4 \
    --edit-dir ./my_edit \
    --language en \
    --num-speakers 2

To load and iterate over word timestamps in Python:

import json
from pathlib import Path

transcript_path = Path("my_edit/transcripts/video.json")
data = json.loads(transcript_path.read_text())

for word_info in data.get("words", []):
    print(f"{word_info['word']}: {word_info['start']}–{word_info['end']} s")

To render subtitles aligned to word timestamps:

from helpers.render import render_subtitles

render_subtitles(
    video_path=Path("my_edit/video.mp4"),
    transcript=data,
    output_path=Path("my_edit/video_with_subs.mp4")
)

Summary

• ElevenLabs Scribe provides word-level timestamps when configured with "timestamps_granularity": "word" in the API request payload.
• The browser-use/video-use repository processes video audio through helpers/transcribe.py, which extracts mono 16kHz WAV files and manages the Scribe API communication.
• The API returns a structured words array containing text content with precise start and end times in seconds.
• Caching mechanisms in the transcription script ensure deterministic, repeatable timestamp results across multiple execution runs.
• Downstream components utilize this granular data for interactive timelines, frame-accurate subtitles, and markdown transcript packaging.

Frequently Asked Questions

What audio format does ElevenLabs Scribe require for optimal word-level timestamp accuracy?

The video-use repository converts input videos to mono 16kHz WAV files using ffmpeg before uploading to Scribe. This specific format, implemented in the extract_audio() function of helpers/transcribe.py, ensures the API receives standardized audio that meets ElevenLabs' processing requirements for precise temporal alignment.

How does the repository handle repeated transcription of the same video file?

The helpers/transcribe.py script implements caching logic at lines 100-108 that checks for existing transcript files before initiating new API calls. If a transcript already exists for the video, the script loads the cached JSON containing the word-level timestamps rather than re-processing the audio, ensuring consistent results and avoiding redundant API usage.

Can Scribe identify different speakers while providing word-level timestamps?

Yes, the API request configures "diarize": "true" alongside the word-level granularity setting. This enables Scribe to attribute each word in the words array to a specific speaker while maintaining precise start and end timestamps, allowing the video-use tools to generate speaker-aware timelines and color-coded transcripts.

What is the exact structure of the timestamp data returned by the Scribe API?

The API returns a JSON object containing a top-level words array where each element is an object with three properties: word (the spoken text as a string), start (the beginning time in seconds as a float), and end (the ending time in seconds as a float). This structure allows direct calculation of word duration and precise positioning on video timelines.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}