# How to Configure Smart Turn Endpointing for Better Turn-Taking

> Learn to configure Smart Turn endpointing with ONNX models to improve turn-taking in your speech-to-speech applications and prevent interruptions. Optimize conversational flow.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-10

---

**Smart Turn endpointing uses a lightweight ONNX model to predict whether a user has finished speaking, allowing the system to delay assistant responses during incomplete turns and prevent conversational interruptions.**

The `huggingface/speech-to-speech` repository implements an advanced turn-taking mechanism called Smart Turn that reduces premature interruptions in voice conversations. By configuring Smart Turn endpointing parameters, you can fine-tune how the system distinguishes between completed and incomplete user utterances. This guide explains the architecture, configuration options, and tuning strategies based on the actual source code implementation.

## Smart Turn Architecture Overview

Smart Turn operates as a secondary analysis layer after the **Silero VAD** detects the end of a speech segment. The system predicts whether the user's utterance is *complete* or *incomplete*, then adjusts the speculative response window accordingly. This flow is exercised in the test suite ([`tests/test_smart_turn.py`](https://github.com/huggingface/speech-to-speech/blob/main/tests/test_smart_turn.py) lines 44‑62).

### Core Components

The implementation consists of three primary components defined across the codebase:

**VAD Handler** ([`src/speech_to_speech/VAD/vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/vad_handler.py)): The orchestration layer that initializes Smart Turn and executes turn-taking logic. The `setup()` method (lines 59-84) creates a `SmartTurnAnalyzer` instance when `smart_turn=True` and stores configuration values for later use.

```python

# src/speech_to_speech/VAD/vad_handler.py (lines 59-84)

def setup(..., smart_turn: bool = True,
          smart_turn_model_path: str | None = None,
          smart_turn_threshold: float = 0.5,
          smart_turn_max_wait_ms: int = 2000,
          smart_turn_incomplete_delay_ms: int = 600,
          smart_turn_cpu_count: int = 1, ...):

```

**Smart Turn Analyzer** ([`src/speech_to_speech/VAD/smart_turn.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/smart_turn.py)): Wraps the ONNX inference model and handles audio preprocessing. The analyzer pads or trims input to a maximum of **8 seconds** (`MAX_AUDIO_SECONDS = 8`) and resamples to **16 kHz** (`MODEL_SAMPLE_RATE = 16000`) before running prediction.

```python

# src/speech_to_speech/VAD/smart_turn.py

class SmartTurnAnalyzer:
    def __init__(..., model_path: str | None = None,
                 threshold: float = 0.5,
                 cpu_count: int = 1, warmup: bool = True):

```

**Argument Classes** ([`src/speech_to_speech/arguments_classes/vad_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/vad_arguments.py)): Exposes all Smart Turn parameters to the CLI and configuration system through the `VADHandlerArguments` dataclass.

## Configuration Parameters

All Smart Turn settings are accessible via the `VADHandlerArguments` class in [`src/speech_to_speech/arguments_classes/vad_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/vad_arguments.py). The following table details each parameter:

| Parameter | Default | Description |
|-----------|---------|-------------|
| `smart_turn` | `True` | Master toggle to enable or disable Smart Turn analysis |
| `smart_turn_threshold` | `0.5` | Probability threshold for classifying a turn as "complete" |
| `smart_turn_max_wait_ms` | `2000` | Maximum milliseconds to keep response speculative when turn is incomplete |
| `smart_turn_incomplete_delay_ms` | `600` | Processing delay added after incomplete predictions to allow speech resumption |
| `smart_turn_cpu_count` | `1` | Number of CPU threads allocated to ONNX Runtime inference |
| `smart_turn_model_path` | `None` | Optional path to a custom ONNX model file |

## Implementation Examples

You can configure Smart Turn endpointing programmatically through the Python API or via command-line arguments when running the pipeline.

### Python API Configuration

Instantiate the `VADHandler` and call `setup()` with your desired Smart Turn parameters. Note that you must provide a `SpeculativeTurnTracker` instance when enabling speculative turns.

```python
from speech_to_speech.VAD.vad_handler import VADHandler
from speech_to_speech.pipeline.speculative_turns import SpeculativeTurnTracker
from threading import Event

# Initialize required tracker

tracker = SpeculativeTurnTracker()

handler = VADHandler()
handler.setup(
    should_listen=Event(),
    speculative_turns=tracker,
    smart_turn=True,
    smart_turn_threshold=0.7,          # Require higher confidence for completion

    smart_turn_max_wait_ms=3000,       # Wait up to 3s for incomplete turns

    smart_turn_incomplete_delay_ms=800,
    smart_turn_cpu_count=2,            # Parallel inference threads

)

```

### CLI Configuration

Pass arguments directly when launching the pipeline from the command line:

```bash
python -m speech_to_speech.cli \
    --smart_turn \
    --smart_turn_threshold 0.6 \
    --smart_turn_max_wait_ms 2500 \
    --smart_turn_incomplete_delay_ms 500 \
    --smart_turn_cpu_count 2

```

## Turn-Taking Logic and Tuning

The method `_smart_turn_timing_ms` (lines 509-542 in [`vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/vad_handler.py)) implements the decision logic that translates model predictions into timing adjustments. This function returns a tuple `(reopen_grace_ms, processing_delay_ms)` that controls the speculative window and downstream processing delays.

### Behavior Modes

The system responds differently based on the `SmartTurnResult`:

- **Complete Turn**: When `result.complete == True`, the handler uses the standard speculative reopen window (default 800 ms) with no additional processing delay.
- **Incomplete Turn**: The system extends the speculative grace period to `smart_turn_max_wait_ms` and adds `smart_turn_incomplete_delay_ms` before starting STT/LLM processing. This allows users to resume speaking without the assistant interrupting.
- **Inference Failure**: Falls back to the default speculative window to maintain conversation flow.

### Tuning Strategies

Adjust these parameters to optimize for your specific use case:

- **Increase `smart_turn_threshold`** (e.g., to 0.7 or 0.8) to make the system more conservative about declaring turns complete. This prevents cut-offs during short pauses but may increase response latency.
- **Raise `smart_turn_max_wait_ms`** for users with slower speech patterns or when running complex downstream pipelines that benefit from longer speculative windows.
- **Lower `smart_turn_incomplete_delay_ms`** (e.g., to 300-400 ms) for faster response times in high-latency environments, though this risks processing utterances that the user intends to continue.
- **Increase `smart_turn_cpu_count`** on multi-core servers to reduce inference latency, though the default single-threaded configuration is optimal for most client-side deployments.

## Summary

- Smart Turn endpointing in `huggingface/speech-to-speech` uses an ONNX model to classify user utterances as complete or incomplete after VAD detection.
- Configure parameters via `VADHandlerArguments` in [`src/speech_to_speech/arguments_classes/vad_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/vad_arguments.py) or through CLI flags.
- Key tuning knobs include `smart_turn_threshold` for completion confidence and `smart_turn_max_wait_ms` for speculative window duration.
- The `_smart_turn_timing_ms` method in [`vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/vad_handler.py) orchestrates turn-taking by adjusting grace periods and processing delays based on model predictions.

## Frequently Asked Questions

### What is the default Smart Turn threshold and when should I change it?

The default threshold is **0.5**, defined in [`src/speech_to_speech/arguments_classes/vad_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/vad_arguments.py). Increase this value to 0.6-0.8 when your application serves users who pause frequently mid-thought, as it requires higher model confidence before committing the assistant's response. Lower values (0.3-0.4) create more responsive but potentially interruptive behavior for fast turn-taking scenarios.

### How does Smart Turn handle incomplete user utterances?

When the analyzer predicts an incomplete turn (probability below threshold), the system activates two protective mechanisms defined in `_smart_turn_timing_ms` (lines 509-542 of [`vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/vad_handler.py)). First, it extends the speculative response window to `smart_turn_max_wait_ms` (default 2000 ms), allowing the user to continue speaking. Second, it introduces a processing delay of `smart_turn_incomplete_delay_ms` (default 600 ms) before sending audio to the STT/LLM pipeline, preventing wasted computation on partial utterances.

### Can I use a custom ONNX model for Smart Turn prediction?

Yes. Pass a file path to the `smart_turn_model_path` parameter in the `VADHandler.setup()` method or via the `--smart_turn_model_path` CLI argument. The custom model must accept audio input preprocessed to 16 kHz sample rate and maximum 8 seconds duration, matching the interface defined in [`src/speech_to_speech/VAD/smart_turn.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/smart_turn.py). The `SmartTurnAnalyzer` handles all resampling and padding automatically.

### What audio preprocessing does the Smart Turn Analyzer perform?

According to [`src/speech_to_speech/VAD/smart_turn.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/VAD/smart_turn.py), the `SmartTurnAnalyzer` resamples all input to **16 kHz** (`MODEL_SAMPLE_RATE = 16000`) and pads or truncates audio to a maximum of **8 seconds** (`MAX_AUDIO_SECONDS = 8`). This standardization ensures compatibility with the ONNX model's expected input dimensions while maintaining real-time performance characteristics suitable for conversational AI applications.