What Is the Role of ElevenLabs Scribe in Video-Use Transcription?

ElevenLabs Scribe serves as the core speech-to-text engine that converts raw video audio into diarized, word-level timestamped transcripts, forming the foundational "Layer 1" that drives video-use's automated editing decisions.

The browser-use/video-use repository leverages ElevenLabs Scribe to transform unstructured video content into structured JSON data that powers AI-driven editing. This integration handles audio extraction, API communication, and intelligent caching to produce the precise transcripts required for identifying speaker changes, silence gaps, and optimal cutting points.

How ElevenLabs Scribe Integrates into the Transcription Pipeline

The transcription workflow follows a four-stage pipeline implemented in helpers/transcribe.py. Each stage prepares and processes audio specifically for the ElevenLabs Scribe API.

Audio Extraction and Standardization

Before uploading to Scribe, video files must conform to specific audio specifications. The extract_audio function uses ffmpeg to convert source videos into mono 16 kHz WAV files. This standardization ensures consistent quality and reduces payload size for the API request while maintaining the fidelity required for accurate speech recognition.

The Scribe API Request Structure

The core API interaction occurs in call_scribe within helpers/transcribe.py (lines 64–82). This function uploads the processed WAV file to https://api.elevenlabs.io/v1/speech-to-text with specific parameters optimized for video editing:

• diarize: true – Enables speaker identification to distinguish between multiple voices
• tag_audio_events: true – Detects non-speech audio elements like music or background noise
• timestamps_granularity: "word" – Provides precise word-level timing essential for cut-point accuracy

Optional parameters include language (ISO language code) and num_speakers to improve diarization accuracy when the speaker count is known in advance.

Response Handling and Caching

Scribe returns a JSON payload containing the transcript text, speaker segments, and detected audio events. The transcribe_one function saves this data to <edit_dir>/transcripts/<video_stem>.json, creating a persistent cache that skips subsequent API calls if the transcript file already exists. This caching mechanism prevents redundant uploads and reduces API costs during iterative editing workflows.

Implementation Details in helpers/transcribe.py

The transcription logic resides in two primary modules that handle both individual and batch processing scenarios.

Single-File Transcription

For processing individual videos, import transcribe_one and load_api_key from helpers/transcribe.py. The function automatically handles audio extraction, API authentication via the ELEVENLABS_API_KEY environment variable, and JSON serialization:

from helpers.transcribe import transcribe_one, load_api_key
from pathlib import Path

video_path = Path("my_take.mp4")
edit_dir   = Path("my_take/edit")
api_key    = load_api_key()               # reads ELEVENLABS_API_KEY from .env

# Performs extraction, uploads to Scribe, and writes the JSON transcript.

transcript_path = transcribe_one(
    video=video_path,
    edit_dir=edit_dir,
    api_key=api_key,
    language="en",          # optional ISO language code

    num_speakers=2,         # optional speaker count to improve diarization

)
print(f"Transcript saved to {transcript_path}")

Batch Processing Workflow

For directory-level operations, helpers/transcribe_batch.py orchestrates parallel transcription using multiple workers. The module reuses transcribe_one while managing concurrency:

from helpers.transcribe_batch import main as batch_main

# Run from the command line:

#   python helpers/transcribe_batch.py /path/to/videos --workers 4

# Equivalent programmatic call:

batch_main()   # parses sys.argv; adapt argv if needed

Direct Scribe API Access

For lower-level control, access the call_scribe and extract_audio functions directly to manipulate audio parameters or inspect raw API responses:

from helpers.transcribe import call_scribe, extract_audio
from pathlib import Path
import tempfile

video = Path("my_take.mp4")
api_key = "YOUR_ELEVENLABS_API_KEY"

with tempfile.TemporaryDirectory() as td:
    wav = Path(td) / "temp.wav"
    extract_audio(video, wav)                     # creates mono 16 kHz WAV

    payload = call_scribe(wav, api_key,
                          language="en",
                          num_speakers=2)
print(payload["words"][:5])  # preview first few words with timestamps

Scribe Output as Layer 1 of the Editing Pipeline

According to the architecture documentation in README.md (lines 73–80), the transcript generated by ElevenLabs Scribe constitutes Layer 1 of the editing pipeline. This compact JSON-derived text (approximately 12 KB per video) provides the structured data necessary for the LLM to reason about editorial decisions without processing raw video frames.

The word-level timestamps and speaker diarization enable the system to identify precise cut points, handle speaker transitions, and detect silence gaps. Visual PNG frames are generated on-demand only for ambiguous sections, making the Scribe transcript the primary input that drives video-use's automated editing intelligence.

Summary

• ElevenLabs Scribe provides the speech-to-text engine that converts video audio into structured JSON transcripts in browser-use/video-use.
• The pipeline in helpers/transcribe.py extracts mono 16 kHz WAV audio, uploads it to https://api.elevenlabs.io/v1/speech-to-text, and caches results to avoid redundant API calls.
• Scribe configuration enables diarization, audio event tagging, and word-level timestamps essential for precise video editing.
• The generated transcript forms Layer 1 of the editing architecture, feeding the LLM with compact text data to drive cutting decisions.

Frequently Asked Questions

What audio format does video-use send to ElevenLabs Scribe?

The system converts all source videos to mono 16 kHz WAV files using ffmpeg through the extract_audio function. This standardization ensures optimal compatibility with the Scribe API while minimizing upload payload size.

How does video-use prevent duplicate API calls for the same video?

The transcribe_one function implements a caching mechanism that checks for existing transcript files at <edit_dir>/transcripts/<video_stem>.json. If the file exists, the function skips the upload entirely, reducing API costs and processing time for repeated runs.

Which Scribe parameters does video-use enable for video editing?

The implementation specifically requests diarization (diarize: true), audio event tagging (tag_audio_events: true), and word-level timestamps (timestamps_granularity: "word"). These features enable the editing pipeline to identify speakers, detect non-speech events, and make precise cut decisions based on exact word timing.

Where should the ElevenLabs API key be configured?

The repository expects the ELEVENLABS_API_KEY environment variable to be defined in a .env file at the project root, as demonstrated in .env.example. The load_api_key function in helpers/transcribe.py reads this value to authenticate API requests.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}