# Implementing Voice Activity Detection Interrupt for Turn-Taking in Speech-to-Speech

> Implement voice activity detection interrupt for real-time turn-taking in speech-to-speech with Hugging Face's VADHandler. Control responsiveness and prevent false triggers.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-09

---

**The Hugging Face speech-to-speech library enables real-time turn-taking through the `VADHandler` class, which detects new user speech during system responses and triggers an interrupt event via the `interrupt_on_new_speech` parameter, configurable through `min_speech_continuation_ms` to balance responsiveness against false triggers.**

The huggingface/speech-to-speech repository provides a modular pipeline for streaming speech-to-speech interactions. Implementing voice activity detection interrupt for turn-taking allows the system to naturally handle conversational overlaps, stopping text-to-speech generation when the user begins speaking again.

## Core Architecture

The turn-taking system relies on three primary components that process audio streams and coordinate state transitions:

| Component | Responsibility | Source Location |
|-----------|----------------|-----------------|
| **`VADIterator`** | Consumes raw audio chunks, runs the VAD model, and yields speech or silence events while tracking utterance state. | [`src/speech_to_speech/VAD/vad_iterator.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/vad_iterator.py) |
| **`VADHandler`** | Orchestrates the iterator, buffers audio, and emits `VADAudio` messages; implements the `_maybe_interrupt` logic for aborting ongoing utterances. | [`src/speech_to_speech/VAD/vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/vad_handler.py) |
| **`SmartTurn`** | Analyzes VAD output alongside LLM responses to predict optimal turn-taking moments and can pre-empt the speaker when confidence is high. | [`src/speech_to_speech/VAD/smart_turn.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/smart_turn.py) |

The pipeline communicates through lightweight message objects. The **`VADAudio`** class, defined in [`src/speech_to_speech/pipeline/messages.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/pipeline/messages.py), carries audio buffers with mode indicators (`"progressive"` for ongoing speech or `"final"` for completed segments) and unique turn identifiers.

## How the Interrupt Mechanism Works

The interrupt logic inside `VADHandler` manages the transition between listening and speaking states through a five-stage process:

1. **Audio Ingestion** – Raw PCM chunks enter the `VADHandler.process` method from the input stream.

2. **VAD Iteration** – The handler delegates to `VADIterator`, which applies a VAD model (such as Silero VAD or a Whisper-based detector) to classify audio as speech or silence.

3. **State Detection** – When the iterator signals speech after a period of silence, the handler checks if a previous utterance remains open (not marked `"final"`).

4. **Interrupt Decision** – If `interrupt_on_new_speech` is enabled and the iterator reports new speech while the previous `VADAudio` is still progressive, the private `_maybe_interrupt` method finalizes the previous segment (setting mode to `"final"`), clears the buffer, and initializes a fresh `VADAudio` with mode `"progressive"`.

5. **Downstream Propagation** – The finalized segment propagates to STT, LLM, and TTS handlers, allowing the system to abort the current generation and process the new user input.

The `min_speech_continuation_ms` parameter prevents spurious interrupts by requiring a minimum pause duration before an utterance can be considered complete, protecting against brief pauses in continuous speech.

## Configuration Parameters

Turn-taking behavior is controlled through **`VADHandlerArguments`** located in [`src/speech_to_speech/arguments_classes/vad_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/vad_arguments.py). Key parameters include:

- **`interrupt_on_new_speech`** – Boolean flag that enables the interrupt mechanism when set to `True`.
- **`min_speech_continuation_ms`** – Integer specifying the minimum silence duration (in milliseconds) required to finalize an utterance before a new interrupt can occur.
- **`vad_model`** – String identifier for the VAD backend (e.g., `"silero_vad"`).

These parameters are passed to `VADHandler` during pipeline construction or direct instantiation.

## Implementation Examples

### Basic Pipeline Integration

Configure the VAD handler through the pipeline constructor to enable interrupts in a complete speech-to-speech system:

```python
from speech_to_speech import SpeechToSpeechPipeline
from speech_to_speech.arguments_classes import VADHandlerArguments

vad_args = VADHandlerArguments(
    vad_model="silero_vad",
    min_speech_continuation_ms=200,
    interrupt_on_new_speech=True,
)

pipeline = SpeechToSpeechPipeline(
    vad_handler_kwargs=vad_args,
    # Additional handlers: stt_handler, llm_handler, tts_handler...

)

def audio_callback(indata, frames, time, status):
    pipeline.process_audio(indata.astype("float32"))

# Pipeline automatically emits finalized VADAudio on interrupt or natural pauses

```

### Direct VADHandler Usage

Interact with the handler directly to observe interrupt behavior on simulated audio streams:

```python
from speech_to_speech.VAD.vad_handler import VADHandler
from speech_to_speech.pipeline.messages import VADAudio
import numpy as np

handler = VADHandler(
    vad_model="silero_vad",
    min_speech_continuation_ms=300,
    interrupt_on_new_speech=True,
)

# Simulated audio: silence, speech, short pause, new speech (triggers interrupt)

audio_chunks = [
    np.zeros(16000, dtype=np.float32),
    np.random.randn(8000).astype(np.float32) * 0.02,
    np.zeros(1600, dtype=np.float32),
    np.random.randn(8000).astype(np.float32) * 0.02,
]

for chunk in audio_chunks:
    msgs = list(handler.process(VADAudio(audio=chunk, mode="progressive")))
    for msg in msgs:
        print(f"Mode: {msg.mode}, Turn ID: {msg.turn_id}")

```

This outputs a `"final"` message for the first speech segment followed by a `"progressive"` message for the interrupting second segment.

### Tuning Interrupt Sensitivity

Adjust the continuation threshold to match conversational dynamics:

```python
vad_args = VADHandlerArguments(
    min_speech_continuation_ms=500,  # Require 500ms silence before finalizing

    interrupt_on_new_speech=True,
)
handler = VADHandler(**vad_args.model_dump())

```

Higher values prevent premature turn-taking in languages with frequent mid-sentence pauses, while lower values increase responsiveness for rapid back-and-forth dialogue.

## Key Source Files

The interrupt implementation spans the following repository locations:

- **[`src/speech_to_speech/VAD/vad_iterator.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/vad_iterator.py)** – Low-level iterator managing VAD model inference and speech/silence state transitions.
- **[`src/speech_to_speech/VAD/vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/vad_handler.py)** – High-level handler containing `_maybe_interrupt` logic and `VADAudio` message generation.
- **[`src/speech_to_speech/VAD/smart_turn.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/smart_turn.py)** – Optional analyzer for LLM-aware turn prediction.
- **[`src/speech_to_speech/pipeline/messages.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/pipeline/messages.py)** – Definition of `VADAudio` dataclass with `mode` and `turn_id` fields.
- **[`src/speech_to_speech/arguments_classes/vad_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/vad_arguments.py)** – Configuration dataclass for VAD parameters.

## Summary

- The **`VADHandler`** class in [`src/speech_to_speech/VAD/vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/vad_handler.py) orchestrates turn-taking through the `_maybe_interrupt` method, which finalizes ongoing utterances when new speech is detected.
- Enable interrupts by setting **`interrupt_on_new_speech=True`** in `VADHandlerArguments`.
- Control sensitivity with **`min_speech_continuation_ms`** to balance between responsive interruption and false triggers during natural pauses.
- The system uses **`VADAudio`** messages with `"progressive"` and `"final"` modes to signal state changes through the pipeline.
- **`SmartTurn`** provides additional intelligence for preemptive turn-taking based on LLM confidence scores.

## Frequently Asked Questions

### What triggers a voice activity detection interrupt in the pipeline?

A VAD interrupt triggers when the `VADIterator` detects new speech activity while the `VADHandler` is still processing an open utterance (mode `"progressive"`) and the `interrupt_on_new_speech` parameter is enabled. The handler's `_maybe_interrupt` method finalizes the current segment and begins buffering the new input.

### How do I prevent the system from interrupting during natural speech pauses?

Increase the **`min_speech_continuation_ms`** value in `VADHandlerArguments`. This parameter defines the minimum silence duration required before an utterance can be finalized, preventing brief pauses from triggering false turn-taking events.

### What is the difference between `VADIterator` and `VADHandler`?

`VADIterator`, located in [`src/speech_to_speech/VAD/vad_iterator.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/vad_iterator.py), performs the low-level audio classification and state tracking. `VADHandler`, in [`src/speech_to_speech/VAD/vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/vad_handler.py), wraps the iterator with buffering logic, interrupt management, and message generation for the broader pipeline.

### Can SmartTurn operate without the VAD interrupt mechanism?

Yes, `SmartTurn` in [`src/speech_to_speech/VAD/smart_turn.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/smart_turn.py) can function as an independent turn-taking strategy by analyzing LLM response confidence and VAD history. However, combining it with `interrupt_on_new_speech` provides the most responsive user experience, allowing both user-driven and system-driven turn transitions.