How video-use Detects and Removes Filler Words and Silence Gaps from Videos

TLDR: video-use processes word-level transcripts from speech-to-text engines like Whisper to filter out hard-coded filler words and detect silence gaps using a threshold-based algorithm, producing clean phrase-level output for video editing workflows.

The browser-use/video-use repository provides post-processing utilities that detect and remove filler words and silence gaps from video transcripts. Operating on Whisper-generated word-level JSON, the library implements a two-stage pipeline that filters disfluencies and identifies pause intervals to create concise phrase boundaries.

Filler Word Detection and Removal

Hard-Coded Filler Vocabulary

The system maintains a predefined set of common filler utterances including "um", "uh", "like", and "you know". This hard-coded list targets typical speech disfluencies that editors typically remove during video post-production.

Case-Insensitive Filtering in transcribe.py

In helpers/transcribe.py, the library iterates through each word object in the transcript and performs case-insensitive comparison against the filler list. When the word field matches an entry, that object is dropped immediately, ensuring fillers never appear in final phrase or subtitle outputs. This filtering occurs early in the transcript-packing pipeline before silence detection begins according to the source code in browser-use/video-use.

# Example: clean a transcript and get silence intervals

from helpers.transcribe import load_transcript   # loads Whisper JSON

from helpers.timeline_view import find_silences

# Load the raw word‑level transcript

words = load_transcript("my_video_transcript.json")   # → list[dict]

# 1️⃣ Remove filler words

FILLERS = {"um", "uh", "like", "you know"}
clean_words = [w for w in words if w["word"].lower() not in FILLERS]

# 2️⃣ Detect silences (gap ≥ 0.4 s)

silences = find_silences(
    words=clean_words,
    start=0.0,
    end=clean_words[-1]["end"],
    threshold=0.4,
)

print("Silence gaps (seconds):", silences)

Silence Gap Detection

The find_silences Algorithm

The find_silences function in helpers/timeline_view.py (line 135) implements the core silence detection logic. It receives the cleaned word list, clip start/end times, and a configurable threshold parameter to identify meaningful pauses in the audio timeline.

Threshold-Based Gap Measurement

The algorithm calculates temporal gaps between consecutive words using the formula next_word.start - cur_word.end. When this difference exceeds the threshold (default 0.4 seconds), the interval is recorded as a silence gap. The function returns tuples of (silence_start, silence_end) representing each detected pause. These intervals serve dual purposes: shading silent portions on the visual timeline and providing natural breakpoints for phrase segmentation.

Transcript Packing and Phrase Segmentation

Grouping Words into Phrases

After silence detection, the group_into_phrases routine in helpers/pack_transcripts.py (line 42) uses the detected gaps to split the transcript into natural phrase-level blocks. Rather than relying on fixed time windows, the library uses these silence intervals as semantic breakpoints, creating clean separation between spoken thoughts.

# Example: pack words into phrase‑level blocks, breaking on silences

from helpers.pack_transcripts import pack_one_file

# The helper automatically groups on silences ≥ 0.5 s (default)

md_output, stats = pack_one_file(
    json_path=Path("my_video_transcript.json"),
    silence_threshold=0.5,
)

print(md_output)   # Markdown where each line corresponds to a phrase

Visual Timeline Integration

The silence intervals returned by find_silences are also used to shade silent portions on the visual timeline, providing editors with clear visual markers of non-speaking sections that may require trimming or transition effects.

Summary

• Filler word removal happens in helpers/transcribe.py using a hard-coded, case-insensitive list of common utterances like "um" and "uh".
• Silence detection occurs in helpers/timeline_view.py via the find_silences function (line 135), which calculates gaps between word timestamps using a default 0.4-second threshold.
• Detected silences generate (silence_start, silence_end) tuples used for both timeline visualization and phrase segmentation.
• The group_into_phrases function in helpers/pack_transcripts.py (line 42) converts word-level data into clean phrase blocks using silence gaps as natural boundaries.

Frequently Asked Questions

What filler words does video-use remove by default?

The library targets common speech disfluencies including "um", "uh", "like", and "you know". The comparison is case-insensitive, ensuring these fillers are caught regardless of capitalization in the Whisper transcript.

How does video-use calculate silence gaps between words?

The find_silences function calculates the temporal difference between the end time of the current word and the start time of the next word. When this gap exceeds the configured threshold (default 0.4 seconds), it is recorded as a silence interval.

Can I adjust the silence detection threshold?

Yes. Both the find_silences function and higher-level utilities like pack_one_file accept a threshold or silence_threshold parameter. You can increase this value to ignore brief pauses or decrease it to catch micro-silences for tighter editing.

Does video-use perform its own speech-to-text transcription?

No. The library expects pre-existing word-level transcripts, typically JSON outputs from OpenAI's Whisper or compatible speech-to-text engines. It focuses exclusively on post-processing these transcripts to remove fillers and detect silence gaps.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}