How to Configure VAD Parameters: Threshold, Minimum Speech Duration, and Smart Turn in Hugging Face Speech-to-Speech

Set VADHandlerArguments with threshold (default 0.5), min_speech_ms (default 384 ms), and min_speech_continuation_ms (default 192 ms), then pass the instance to S2SPipeline via vad_handler_kwargs.

The Hugging Face speech-to-speech repository exposes all Voice Activity Detection (VAD) settings through a centralized dataclass. Understanding how these parameters interact lets you tune real-time speech segmentation for latency, noise robustness, or conservative turn-taking.

VADHandlerArguments: The Complete Parameter Reference

Located in src/speech_to_speech/arguments_classes/vad_arguments.py, the VADHandlerArguments dataclass defines every configurable VAD setting:

• threshold (float, default 0.5): Probability cutoff from the Silero VAD model. Chunks scoring above this value are classified as speech.
• smart_turn_threshold (float, default 0.5): Controls the Smart Turn module's pause-based turn finalization. Higher values delay finalization on ambiguous silences.
• min_speech_ms (int, default 384): Minimum duration (in milliseconds) for a speech segment to be emitted as a valid turn. Shorter segments are discarded or buffered.
• min_speech_continuation_ms (int, default 192): Hysteresis window for reopening "soft-ended" turns. If speech resumes within this window, it merges with the previous segment. Range is clamped to [100, min_speech_ms]; set to 0 to disable splitting entirely.

These values flow into VADHandler (src/speech_to_speech/VAD/vad_handler.py), which orchestrates the underlying VADIterator (src/speech_to_speech/VAD/vad_iterator.py) and SmartTurn (src/speech_to_speech/VAD/smart_turn.py).

How VAD Parameters Interact at Runtime

Threshold Application

The VADIterator receives threshold from VADHandler (see lines 199–201 of vad_handler.py). For each audio chunk, Silero's model outputs a speech probability. The iterator marks speech when probability >= threshold, with a small hysteresis (threshold - 0.15) to smooth speech-to-silence transitions (lines 147–153 of vad_iterator.py).

Minimum Speech Duration Enforcement

After raw VAD decisions, VADHandler performs fragment stitching:

1. Consecutive speech fragments are collected.
2. Segments must satisfy min_speech_ms to be finalized.
3. If multiple sub-threshold fragments occur within min_speech_continuation_ms, they merge into a single turn.

This design enables soft-ended turns—pauses that don't immediately terminate speech if the speaker resumes quickly.

Smart Turn Finalization

The SmartTurn module (lines using smart_turn_threshold in smart_turn.py) evaluates whether a detected pause represents genuine turn completion. When the probabilistic score exceeds smart_turn_threshold, the handler finalizes the turn early; otherwise, it waits for more audio.

Code Examples: Configuring VAD Parameters

Default Configuration

from speech_to_speech.arguments_classes.vad_arguments import VADHandlerArguments
from speech_to_speech.s2s_pipeline import S2SPipeline

vad_args = VADHandlerArguments()  # threshold=0.5, min_speech_ms=384, ...

pipeline = S2SPipeline(
    vad_handler_kwargs=vad_args,
    # ... STT, LLM, TTS arguments ...

)
pipeline.run()

Aggressive Detection (Low Latency, Noisy Environments)

vad_args = VADHandlerArguments(
    threshold=0.35,                    # Lower confidence requirement

    min_speech_ms=200,                 # Accept very short utterances

    min_speech_continuation_ms=100,    # Quick turn reopening

    smart_turn_threshold=0.3,          # Fast turn finalization

)

pipeline = S2SPipeline(vad_handler_kwargs=vad_args, ...)
pipeline.run()

Conservative Detection (Quiet, Clean Audio)

vad_args = VADHandlerArguments(
    threshold=0.7,                     # High confidence requirement

    min_speech_ms=500,                 # Ignore brief noises

    min_speech_continuation_ms=300,    # Longer merge window

    smart_turn_threshold=0.6,          # Patient turn finalization

)

pipeline = S2SPipeline(vad_handler_kwargs=vad_args, ...)
pipeline.run()

Key Source Files Reference

File	Purpose
src/speech_to_speech/arguments_classes/vad_arguments.py	VADHandlerArguments dataclass with all configurable VAD parameters
src/speech_to_speech/VAD/vad_handler.py	Core VAD logic; enforces duration constraints and manages turn state
src/speech_to_speech/VAD/vad_iterator.py	Silero VAD wrapper; applies raw threshold with hysteresis
src/speech_to_speech/VAD/smart_turn.py	Probabilistic turn finalization using smart_turn_threshold
src/speech_to_speech/s2s_pipeline.py	Pipeline orchestrator; wires arguments into VADHandler

Summary

• Configure VAD parameters by instantiating VADHandlerArguments and passing it to S2SPipeline as vad_handler_kwargs.
• threshold controls raw speech detection sensitivity at the Silero model level.
• min_speech_ms and min_speech_continuation_ms govern how fragments are stitched into final turns.
• smart_turn_threshold adjusts pause-based turn finalization for more natural conversational flow.
• All parameters are validated and applied in vad_handler.py, with direct updates to vad_iterator.py at runtime.

Frequently Asked Questions

What happens if I set min_speech_continuation_ms to 0?

Setting min_speech_continuation_ms=0 disables the soft-ended turn mechanism. The handler uses only min_speech_ms for segmentation, forcing strict turn boundaries without reopening capabilities.

How do I make the VAD more sensitive to quiet speech?

Lower threshold (e.g., 0.3–0.4) and reduce min_speech_ms (e.g., 200–250 ms). This captures lower-confidence fragments and shorter utterances, though it may increase false activations in noisy conditions.

What's the difference between threshold and smart_turn_threshold?

threshold filters raw VAD probabilities from the Silero model—it's a binary speech/silence decision. smart_turn_threshold operates on the Smart Turn module's probabilistic output, deciding when a pause is long enough to finalize a conversational turn.

Where are these parameters actually enforced in the codebase?

VADHandlerArguments is defined in vad_arguments.py. Runtime enforcement happens in vad_handler.py (duration constraints, fragment stitching), vad_iterator.py (threshold application), and smart_turn.py (turn finalization logic), as implemented in huggingface/speech-to-speech.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}