# How to Configure Smart Turn Endpointing for Voice Activity Detection in Hugging Face Speech-to-Speech

> Learn to configure Smart Turn endpointing for voice activity detection in Hugging Face Speech-to-Speech. Adjust VAD parameters for better speech processing.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-07

---

**Smart Turn endpointing is controlled via six configuration parameters that adjust the completeness threshold, wait times, and inference resources for a post-VAD classifier in [`speech_to_speech/VAD/smart_turn.py`](https://github.com/huggingface/speech-to-speech/blob/main/speech_to_speech/VAD/smart_turn.py).**

## What Is Smart Turn Endpointing?

**Smart Turn** is an optional post-VAD classifier that determines whether a spoken utterance is *complete* or *incomplete*. It integrates directly into the voice activity detection pipeline in the [huggingface/speech-to-speech](https://github.com/huggingface/speech-to-speech) repository to improve turn-taking accuracy in real-time conversations.

When enabled, Smart Turn affects two critical timing behaviors:

- **Speculative reopen grace** — how long the assistant's response remains speculative after speech ends
- **Processing delay** — an optional pause before STT/LLM processing when the turn is flagged as incomplete

Smart Turn is **enabled by default** in recent versions of the framework.

## Smart Turn Architecture

The system comprises three interconnected components:

| Component | File Location | Primary Responsibility |
|-----------|---------------|------------------------|
| **SmartTurnAnalyzer** | [`speech_to_speech/VAD/smart_turn.py`](https://github.com/huggingface/speech-to-speech/blob/main/speech_to_speech/VAD/smart_turn.py) | Loads the ONNX model, runs inference, returns `SmartTurnResult` with `complete`, `probability`, and `inference_ms` fields |
| **VADHandler** | [`speech_to_speech/VAD/vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/speech_to_speech/VAD/vad_handler.py) | Orchestrates VAD detection, accumulates audio, invokes Smart Turn timing logic via `_smart_turn_timing_ms()` |
| **VADHandlerArguments** | [`speech_to_speech/arguments_classes/vad_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/speech_to_speech/arguments_classes/vad_arguments.py) | Exposes all CLI and programmatic configuration knobs |

The inference flow works as follows:

1. VAD detects speech end via `VADIterator`
2. Final segment triggers `_smart_turn_timing_ms()` in `VADHandler` (lines 15-22)
3. If `SmartTurnAnalyzer` exists, `predict()` runs ONNX inference
4. Based on `result.complete`, the handler returns timing values:
   - **Complete turn**: `speculative_reopen_ms` (default 800 ms)
   - **Incomplete turn**: `smart_turn_max_wait_ms` (default 2000 ms) plus `processing_delay_ms` (clamped to `smart_turn_incomplete_delay_ms`)

These values schedule the speculative reopen grace (lines 71-74) and optionally delay downstream processing (lines 81-82).

## Smart Turn Configuration Parameters

All parameters are defined in [`vad_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/vad_arguments.py) (lines 82-115) and passed to `VADHandler.setup()`:

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `smart_turn` | bool | `True` | Master enable/disable switch |
| `smart_turn_model_path` | `str` \| `None` | `None` | Path to custom ONNX model; auto-downloads v3.2 from `pipecat-ai/smart-turn-v3` if omitted |
| `smart_turn_threshold` | `float` | `0.5` | Probability threshold for declaring turn complete (0.0–1.0) |
| `smart_turn_max_wait_ms` | `int` | `2000` | Maximum grace period for incomplete turns in milliseconds |
| `smart_turn_incomplete_delay_ms` | `int` | `600` | Additional STT/LLM delay after incomplete prediction; clamped to `smart_turn_max_wait_ms` |
| `smart_turn_cpu_count` | `int` | `1` | CPU threads for ONNX Runtime inference |

## CLI Configuration Examples

### Basic Enable with Custom Threshold

```bash
python -m speech_to_speech \
  --smart_turn \
  --smart_turn_threshold 0.7 \
  --smart_turn_max_wait_ms 2500

```

*Raises the completeness threshold to reduce false positives in backchanneling scenarios.*

### Production Tuning with Multi-Threading

```bash
python -m speech_to_speech \
  --smart_turn \
  --smart_turn_threshold 0.6 \
  --smart_turn_max_wait_ms 3000 \
  --smart_turn_incomplete_delay_ms 800 \
  --smart_turn_cpu_count 4

```

*`smart_turn_cpu_count` increases inference throughput on multi-core systems without affecting latency for single concurrent streams.*

## Programmatic Configuration

### Standard Setup via VADHandlerArguments

```python
from speech_to_speech.VAD.vad_handler import VADHandler
from speech_to_speech.arguments_classes.vad_arguments import VADHandlerArguments
import threading

args = VADHandlerArguments(
    smart_turn=True,
    smart_turn_threshold=0.6,
    smart_turn_max_wait_ms=3000,
    smart_turn_incomplete_delay_ms=700,
    smart_turn_cpu_count=4,
)

handler = VADHandler()
handler.setup(
    should_listen=threading.Event(),
    speculative_turns=speculative_tracker,
    thresh=args.thresh,
    sample_rate=args.sample_rate,
    smart_turn=args.smart_turn,
    smart_turn_model_path=args.smart_turn_model_path,
    smart_turn_threshold=args.smart_turn_threshold,
    smart_turn_max_wait_ms=args.smart_turn_max_wait_ms,
    smart_turn_incomplete_delay_ms=args.smart_turn_incomplete_delay_ms,
    smart_turn_cpu_count=args.smart_turn_cpu_count,
)

```

*The `setup()` signature mirrors lines 77-84 in [`vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/vad_handler.py), ensuring all timing parameters propagate correctly.*

### Direct SmartTurnAnalyzer Instantiation

```python
from speech_to_speech.VAD.smart_turn import SmartTurnAnalyzer

handler.smart_turn_analyzer = SmartTurnAnalyzer(
    model_path="/path/to/custom-smart-turn.onnx",
    threshold=0.55,
    cpu_count=2,
)

```

*Useful for A/B testing model variants. The analyzer loads weights at initialization (lines 60-73 in [`smart_turn.py`](https://github.com/huggingface/speech-to-speech/blob/main/smart_turn.py)).*

### Disabling Smart Turn Entirely

```python
handler.setup(
    should_listen=should_listen_event,
    speculative_turns=tracker,
    thresh=0.5,
    sample_rate=16000,
    smart_turn=False,  # Disables analyzer creation

)

```

*When `smart_turn=False`, lines 103-108 in [`vad_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/vad_handler.py) skip analyzer instantiation and fall back to fixed `speculative_reopen_ms` timing.*

## Selecting Appropriate Threshold Values

| Scenario | Recommended Threshold | Rationale |
|----------|----------------------|-----------|
| Fast-paced dialogue | `0.3`–`0.4` | Aggressive turn completion reduces perceived latency |
| Professional meetings | `0.5`–`0.6` | Balanced handling of pauses and interruptions |
| Technical support calls | `0.6`–`0.8` | Conservative completion avoids cutting off thinking time |
| Children's speech | `0.4`–`0.5` | Accounts for irregular pause patterns |

Higher thresholds reduce premature turn-taking but may increase perceived response latency.

## Custom Model Deployment

To use a fine-tuned Smart Turn model without modifying source:

1. Train or export to ONNX format compatible with the expected input tensor shape
2. Specify path via `--smart_turn_model_path` or `VADHandlerArguments`
3. Validate with `SmartTurnAnalyzer.predict()` on sample audiochunks before production deployment

The default v3.2 model is fetched from Hugging Face Hub at `pipecat-ai/smart-turn-v3` when no custom path is provided.

## Summary

- **Smart Turn endpointing** improves VAD accuracy by classifying turn completeness using an ONNX neural network
- **Configuration** happens through six parameters in `VADHandlerArguments`, defined in [`vad_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/vad_arguments.py)
- **Enablement** is controlled by the boolean `smart_turn` flag; all other parameters are ignored when `False`
- **Threshold tuning** (`smart_turn_threshold`) directly trades off latency against interruption risk
- **Resource allocation** (`smart_turn_cpu_count`) scales inference without pipeline restructuring

## Frequently Asked Questions

### What happens when Smart Turn predicts an incomplete turn?

The `VADHandler` extends the speculative reopen window to `smart_turn_max_wait_ms` and adds `processing_delay_ms` (the lesser of `smart_turn_incomplete_delay_ms` and `smart_turn_max_wait_ms`) before starting STT/LLM processing. This gives the user time to continue speaking without the assistant interrupting.

### Can I use Smart Turn without an internet connection?

Yes. Download the default v3.2 model once, then reference it locally via `--smart_turn_model_path /local/path/to/model.onnx`. The `SmartTurnAnalyzer` loads entirely from local filesystem after initialization.

### How does `smart_turn_cpu_count` affect performance?

It controls intra-op parallelism in ONNX Runtime. Values above 4 rarely benefit single-stream inference; instead, increase this parameter when running multiple concurrent `VADHandler` instances on the same host. The default of `1` minimizes thread contention for typical deployments.

### Where is the Smart Turn model stored when auto-downloaded?

The `pipecat-ai/smart-turn-v3` model is cached via the Hugging Face Hub library, typically in `~/.cache/huggingface/hub/`. Set `HF_HOME` environment variable to relocate this cache if disk space is constrained.