How Video Cut Candidates Are Determined from Word Boundaries and Silence Gaps

Video cut candidates are generated by analyzing word-level timestamps from ElevenLabs Scribe, identifying gaps between words that exceed a configurable silence threshold (default 0.5s), and grouping consecutive words into phrases at these natural boundaries.

The browser-use/video-use repository implements a precise algorithm for identifying optimal cut points in video editing workflows. By processing transcript data at the word level, the system detects natural pauses and speaker changes to create editable cut candidates that respect spoken content rhythms.

Word-Level Transcription Data

The foundation of cut detection relies on word-level timestamps produced by ElevenLabs Scribe. When a video is processed via helpers/transcribe.py (lines 29-69), the resulting transcript contains a granular list of entries including:

• Words with start, end, and speaker_id fields
• Audio events marking non-speech sounds
• Spacing entries representing gaps between consecutive words

Each word object provides millisecond-precision timing data that enables the system to calculate exact durations between utterances. This raw timeline data serves as the input for all subsequent cut candidate calculations.

Detecting Silence Gaps in Timeline View

The find_silences function in helpers/timeline_view.py (lines 135-147) scans word lists to identify potential cut points based on temporal gaps. The algorithm specifically examines spacing entries—the intervals between successive words—to detect silence:

• A spacing entry's duration is calculated as end - start
• If this duration exceeds the silence threshold (default 0.5 seconds), the interval is classified as a silence gap
• These gaps become primary cut candidates in the timeline visualization

The function accepts configurable parameters including start, end, and threshold times, allowing editors to adjust sensitivity for different speaking styles or content types.

Grouping Words into Phrases

The group_into_phrases function in helpers/pack_transcripts.py (lines 88-99) implements the core logic for organizing words into coherent segments. This function walks the word list and flushes the current phrase when encountering:

1. Silence gaps ≥ the threshold duration
2. Speaker changes (detected via speaker_id transitions)
3. Large temporal gaps from the previous kept token

This phrase grouping ensures that cut candidates align with natural linguistic boundaries rather than arbitrary time intervals. Each phrase start and end point represents a viable cut location that maintains semantic coherence.

Generating Cut Candidates for Editing

The system emits cut candidates at two granularity levels:

• Phrase boundaries—start and end timestamps of each grouped phrase
• Silence intervals—specific start/end pairs for gaps exceeding the threshold

In the timeline UI, these candidates render as:

• Word markers at each word start for fine-grained navigation
• Shaded silence intervals highlighting obvious pause regions for quick cutting

The words_in_range and find_silences functions provide the data structures powering this visualization, enabling editors to snap cuts to precise word boundaries or select broader silence-based transitions.

Code Examples

# Load a transcript and compute phrase cut points

from helpers.pack_transcripts import group_into_phrases
import json
from pathlib import Path

transcript = json.loads(Path("edit/transcripts/video.json").read_text())
words = transcript["words"]

# Use the default 0.5s silence threshold

phrases = group_into_phrases(words)

# Each phrase gives a start/end cut candidate

for p in phrases:
    print(f"Cut from {p['start']:.2f}s to {p['end']:.2f}s – {p['text']}")

# Find silence intervals for UI shading

from helpers.timeline_view import find_silences, words_in_range
from pathlib import Path

transcript_path = Path("edit/transcripts/video.json")
words = words_in_range(transcript_path, start=0.0, end=1800.0)   # whole video

silences = find_silences(words, start=0.0, end=1800.0, threshold=0.4)

for start, end in silences:
    print(f"Silence from {start:.2f}s to {end:.2f}s")

Summary

• Word-level timestamps from ElevenLabs Scribe provide the temporal foundation for cut detection in helpers/transcribe.py
• Silence detection occurs in helpers/timeline_view.py via the find_silences function, identifying gaps exceeding the 0.5s default threshold
• Phrase grouping in helpers/pack_transcripts.py organizes words into coherent segments at silence boundaries and speaker changes
• Cut candidates combine phrase starts and silence intervals to provide natural, editable break points in the video timeline

Frequently Asked Questions

What is the default silence threshold for detecting cut candidates?

The default silence threshold is 0.5 seconds (500 milliseconds). This value is configurable via the threshold parameter in find_silences and group_into_phrases, allowing adjustment for faster-paced dialogue or more deliberate speech patterns.

How does the system handle speaker changes when determining cut points?

Speaker transitions automatically trigger phrase boundaries regardless of the silence threshold. The group_into_phrases function monitors the speaker_id field in word entries and flushes the current phrase whenever a different speaker ID is encountered, ensuring cuts respect conversational turn-taking.

Which source files contain the core cut detection logic?

The primary files are:

• helpers/timeline_view.py — Contains find_silences (lines 135-147) for gap detection
• helpers/pack_transcripts.py — Contains group_into_phrases (lines 88-99) for phrase assembly
• helpers/transcribe.py — Handles ElevenLabs Scribe integration (lines 29-69) for initial word-level timestamp generation

Can cut candidates be generated for specific time ranges rather than full videos?

Yes. Both words_in_range and find_silences accept start and end parameters (in seconds) to limit processing to specific segments. This enables partial timeline rendering and focused editing on particular sections without loading the entire transcript into memory.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}