ElevenLabs Scribe Word-Level Timestamps and Speaker Diarization in video-use

ElevenLabs Scribe delivers precise word-level timestamps and speaker diarization when invoked through the video-use repository by configuring timestamps_granularity: "word" and diarize: "true" in the API request payload.

The browser-use/video-use repository integrates ElevenLabs Scribe to extract detailed transcripts from video files. By leveraging specific payload parameters in helpers/transcribe.py, the toolkit captures exact timing for every word and segments speech by individual speakers, enabling downstream applications like subtitle generation and timeline editing.

How Word-Level Timestamps Are Enabled

In helpers/transcribe.py, the transcription request explicitly requests granular timing data. The timestamps_granularity parameter is hardcoded to "word" at lines 68-69, ensuring the API returns start and end times for each individual token rather than sentence or paragraph-level timestamps.

# Payload construction in helpers/transcribe.py (lines 65-73)

data = {
    "model_id": "scribe_v1",
    "diarize": "true",                    # Enable speaker diarization

    "tag_audio_events": "true",           # Mark music, silence, etc.

    "timestamps_granularity": "word",     # Word-level precision

    "language_code": language,            # Optional: ISO 639-1 code

    "num_speakers": str(num_speakers)     # Optional: speaker count hint

}

The API returns a words array containing objects with text, start, and end properties, providing timing precision essential for frame-accurate video editing.

Speaker Diarization Capabilities

Speaker diarization is activated by setting diarize to "true" in the request payload at lines 65-66 of helpers/transcribe.py. This instructs Scribe to identify and separate different speakers throughout the audio track.

Optimizing Detection with Speaker Count Hints

When the number of participants is known, the optional num_speakers parameter (lines 72-73) improves diarization accuracy by providing the API with a speaker count hint. The response includes a speaker_segments array, where each entry contains a speaker ID and the corresponding time range.

# Example speaker segment structure

{
    "speaker": "Speaker 1",
    "start": 0.5,
    "end": 4.2
}

Additional Scribe Features

Beyond core transcription, the repository configures Scribe to tag audio events and handle language selection:

• Audio event tagging: tag_audio_events is set to "true" (lines 67-68), adding markers for non-speech events like music or silence
• Language selection: The optional language_code field (lines 70-71) specifies the transcription language when known, improving accuracy for non-English content

Processing Pipeline and Output Structure

The transcription workflow begins with audio extraction. The extract_audio function in helpers/transcribe.py converts the source video to a mono 16kHz WAV file using ffmpeg, optimizing the input for Scribe's processing requirements.

The file is then posted to https://api.elevenlabs.io/v1/speech-to-text with the configured payload. The API response is serialized to <edit_dir>/transcripts/<video_stem>.json, containing two critical arrays:

• words: Ordered list of transcribed tokens with precise timing
• speaker_segments: Chronological list of speaker turns with attribution

Implementation Examples

Transcribing a Single Video

Use the transcribe_one function to process individual files with full timestamp and diarization support:

from pathlib import Path
from helpers.transcribe import transcribe_one, load_api_key

video_path = Path("interview.mp4")
edit_dir = Path("interview-assets")
api_key = load_api_key()  # Reads ELEVENLABS_API_KEY from .env

# Request English transcript with speaker diarization

transcript_path = transcribe_one(
    video=video_path,
    edit_dir=edit_dir,
    api_key=api_key,
    language="en",
    num_speakers=2
)

Parsing Word-Level Timestamps and Speaker Data

Access the granular data from the generated JSON file:

import json

with open(transcript_path) as f:
    data = json.load(f)

# Iterate word-level timestamps

for word in data.get("words", []):
    print(f"[{word['start']:.2f}s - {word['end']:.2f}s] {word['text']}")

# Display diarized speaker segments

for segment in data.get("speaker_segments", []):
    print(f"Speaker {segment['speaker']}: {segment['start']:.2f}s - {segment['end']:.2f}s")

Batch Processing Multiple Videos

Process entire directories using the batch script, which inherits the same capabilities:

python helpers/transcribe_batch.py ./video-folder \
    --workers 4 \
    --language en \
    --num-speakers 2

The transcribe_batch.py wrapper parallelizes calls to transcribe_one, maintaining identical payload configurations for word-level timestamps and speaker diarization across all files.

Summary

• Word-level precision is enforced by setting timestamps_granularity to "word" in helpers/transcribe.py, enabling frame-accurate timing data for every token.
• Speaker diarization activates when diarize is set to "true", with optional num_speakers hints improving accuracy at lines 72-73.
• Audio preprocessing converts video to mono 16kHz WAV via ffmpeg in the extract_audio function before API submission.
• Structured output saves to <edit_dir>/transcripts/<video_stem>.json, containing both words and speaker_segments arrays for downstream tooling.

Frequently Asked Questions

How accurate are ElevenLabs Scribe word-level timestamps?

According to the video-use implementation, Scribe returns precise start and end times in seconds for each word in the words array. This granularity supports frame-level synchronization for subtitle generation and video editing workflows, though exact millisecond precision depends on the audio quality and Scribe's model version.

Can I disable speaker diarization while keeping word timestamps?

Yes. While the repository defaults to diarize: "true" at lines 65-66 of helpers/transcribe.py, you can modify the payload to set "diarize": "false" while maintaining timestamps_granularity: "word". This returns word-level timestamps without speaker attribution, useful for single-speaker content or when speaker identification is unnecessary.

What audio format does the repository send to ElevenLabs Scribe?

The extract_audio function in helpers/transcribe.py processes source videos into mono 16kHz WAV files using ffmpeg before transmission. This conversion ensures compatibility with Scribe's requirements and optimizes processing speed and accuracy compared to sending compressed video formats directly.

How do I handle transcripts for videos with unknown speaker counts?

When the number of speakers is uncertain, omit the num_speakers parameter when calling transcribe_one. The code at lines 70-73 conditionally adds this field only when provided, allowing Scribe to automatically detect speaker changes without the hint. However, providing an accurate count when known improves diarization accuracy for crowded or noisy audio.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}