# How to Implement Barge-In to Interrupt Assistant Responses in the Hugging Face Speech-to-Speech Repository

> Implement barge-in to interrupt assistant responses in Hugging Face Speech-to-Speech. Learn how VAD, runtime flags, and control messages ensure clean turn transitions for seamless interaction.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-07

---

**Barge-in is implemented through three coordinated mechanisms: a runtime configuration flag (`RuntimeConfig.interrupt_response_enabled`), VAD-triggered output queue flushing, and `PipelineControlMessage` signals that guarantee clean turn transitions.**

The Hugging Face **speech-to-speech** repository provides real-time conversational AI with natural barge-in support—the ability for users to interrupt the assistant mid-response. This implementation aligns with the OpenAI Realtime API specification while providing flexible runtime control over interruption behavior.

---

## Runtime Configuration: The `interrupt_response_enabled` Flag

The central control for barge-in resides in [`src/speech_to_speech/api/openai_realtime/runtime_config.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/runtime_config.py). The `RuntimeConfig` class exposes `interrupt_response_enabled` as a property that reads from the session's `turn_detection` configuration.

```python

# src/speech_to_speech/api/openai_realtime/runtime_config.py

@property
def interrupt_response_enabled(self) -> bool:
    """Whether barge‑in should cancel an active response."""
    …
    return val if val is not None else True

```

This property defaults to `True`, matching the OpenAI Realtime API default. The flag is populated from the `interrupt_response` field in the session creation or update request.

### Client-Side Configuration

Enable barge-in when creating a Realtime session:

```json
{
  "audio": {
    "input": {
      "turn_detection": {
        "type": "server_vad",
        "interrupt_response": true
      }
    }
  }
}

```

Or configure programmatically in Python:

```python
from speech_to_speech.api.openai_realtime.runtime_config import RuntimeConfig
from openai.types.realtime.realtime_audio_config_input import RealtimeAudioConfigInput
from openai.types.realtime.realtime_audio_config import RealtimeAudioConfig
from openai.types.realtime.realtime_session_create_request import RealtimeSessionCreateRequest

session = RealtimeSessionCreateRequest(
    type="realtime",
    audio=RealtimeAudioConfig(
        input=RealtimeAudioConfigInput(
            turn_detection={"type": "server_vad", "interrupt_response": True}
        )
    ),
)

runtime_cfg = RuntimeConfig()
runtime_cfg.apply_session_update(session)

```

---

## VAD Handling: Detecting and Acting on Interruptions

The voice activity detector in [`src/speech_to_speech/VAD/vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/vad_handler.py) monitors incoming audio and triggers barge-in logic when `interrupt_response_enabled` is active. When new user speech is detected during an ongoing assistant response, the handler flushes the output queue to immediately stop the text-to-speech generation.

```python

# src/speech_to_speech/VAD/vad_handler.py

if runtime_config.interrupt_response_enabled and new_user_speech:
    output_queue.flush()   # abort current TTS output

```

This queue flush discards in-flight audio frames without corrupting conversation state. The VAD handler preserves critical session-ending signals to ensure the pipeline remains coherent.

---

## Pipeline Control Messages for Clean Turn Transitions

Barge-in requires more than queue flushing—it needs explicit pipeline coordination. The [`src/speech_to_speech/pipeline/control.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/pipeline/control.py) module defines `PipelineControlMessage` and the `SESSION_END` constant:

```python

# src/speech_to_speech/pipeline/control.py

SESSION_END = PipelineControlMessage(ControlKind.SESSION_END)

```

When the VAD handler detects a barge-in, it emits this control message to force the current turn to terminate. This prevents race conditions where partial responses might leak into the next turn.

---

## End-to-End Barge-In Flow

The complete interruption mechanism operates as follows:

1. **Session initialization** – Client sets `interrupt_response: true` in the `turn_detection` configuration
2. **Assistant response** – TTS audio streams to the client through the output queue
3. **User interruption** – VAD detects new speech while response is active
4. **Queue flush** – `runtime_config.interrupt_response_enabled` check passes, `output_queue.flush()` executes
5. **Control signal** – `SESSION_END` message enqueued to finalize the aborted turn
6. **New turn begins** – Pipeline accepts fresh user audio for the next response

---

## Testing Barge-In Behavior

The repository includes tests verifying that barge-in clears state correctly. This pattern validates the flush mechanism preserves essential control messages:

```python
def test_barge_in_flush_preserves_session_end():
    # ...setup pipeline, start a response...

    # Simulate user speaking while response is playing

    pipeline.vad_handler.on_audio_packet(user_speech_packet)
    # VAD will notice `interrupt_response_enabled` and flush the TTS queue

    assert not pipeline.output_queue.has_pending_tts()
    # SESSION_END control message is still present

    assert any(msg.kind == ControlKind.SESSION_END for msg in pipeline.control_messages)

```

---

## Key Implementation Files

| File | Purpose |
|------|---------|
| [`src/speech_to_speech/api/openai_realtime/runtime_config.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/runtime_config.py) | Defines `interrupt_response_enabled` property and session configuration |
| [`src/speech_to_speech/VAD/vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/vad_handler.py) | Detects voice activity and conditionally flushes output queue |
| [`src/speech_to_speech/pipeline/control.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/pipeline/control.py) | Implements `PipelineControlMessage` and `SESSION_END` signal |
| [`src/speech_to_speech/s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py) | Orchestrates VAD, LLM, and TTS components |
| [`tests/openai_realtime/test_websocket_router.py`](https://github.com/huggingface/speech-to-speech/blob/main/tests/openai_realtime/test_websocket_router.py) | Validates barge-in state cleanup |

---

## Summary

- **Enable barge-in** by setting `interrupt_response: true` in the `turn_detection` configuration
- **Runtime check** occurs via `RuntimeConfig.interrupt_response_enabled` in [`runtime_config.py`](https://github.com/huggingface/speech-to-speech/blob/main/runtime_config.py)
- **Immediate response halt** is achieved by `output_queue.flush()` in [`vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/vad_handler.py)
- **State consistency** is guaranteed through `SESSION_END` control messages from [`control.py`](https://github.com/huggingface/speech-to-speech/blob/main/control.py)
- **Default behavior** matches OpenAI Realtime API: interruptions enabled unless explicitly disabled

---

## Frequently Asked Questions

### How do I disable barge-in for my speech-to-speech deployment?

Set `interrupt_response: false` in the `turn_detection` configuration when creating or updating the session. The `RuntimeConfig.interrupt_response_enabled` property will return `False`, and the VAD handler will ignore incoming speech during active responses.

### What happens to audio already queued when a user barges in?

The `output_queue.flush()` call in [`vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/vad_handler.py) discards all pending TTS frames immediately. Any audio already transmitted to the client may continue playing locally, but no new frames are generated or sent.

### Does barge-in affect conversation history or context?

No. The interruption mechanism only affects the current turn's output generation. The `SESSION_END` control message ensures proper turn boundaries without truncating the conversation history sent to the language model.

### Can I customize the VAD sensitivity for barge-in detection?

The repository uses the configured VAD parameters from the OpenAI Realtime session. Adjust `silence_duration_ms` and other `turn_detection` fields to tune how quickly speech triggers an interruption.