# How to Configure Session Instructions, Tools, and Voice Settings at Runtime in Hugging Face Speech-to-Speech

> Configure Hugging Face speech-to-speech runtime session instructions tools and voice settings dynamically using typed dataclass arguments and ParsedArguments for flexible STT LLM and TTS handler assembly.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-03

---

**The Hugging Face speech-to-speech library enables runtime session configuration through typed dataclass arguments that populate a `ParsedArguments` object, allowing dynamic assembly of STT, LLM, and TTS handlers via `S2SPipeline.from_dict()` or CLI parsing.**

The huggingface/speech-to-speech repository implements a modular pipeline architecture where session configuration is decoupled from processing logic. This design allows you to adjust voice settings, language model instructions, and available tools at runtime without modifying the underlying handler implementations in `src/speech_to_speech/`.

## Understanding the Argument Layer Architecture

The configuration system rests on a typed argument layer that uses Python dataclasses to define valid parameters for every pipeline component.

### Typed Configuration Classes

Each handler exposes a dedicated arguments class in `src/speech_to_speech/arguments_classes/`. For example, `WhisperSTTHandlerArguments` configures speech-to-text parameters, while `Qwen3TTSHandlerArguments` controls voice synthesis settings. These dataclasses integrate with `HfArgumentParser` to validate inputs before the pipeline starts.

The `LanguageModelHandlerArguments` class manages session instructions and tool definitions for the LLM component, allowing you to specify system prompts and function calling capabilities programmatically.

### ParsedArguments and Handler Graph Assembly

The `S2SPipeline` class in [`src/speech_to_speech/s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py) collects all argument instances into a single `ParsedArguments` object. During initialization, the pipeline inspects this object to instantiate concrete handlers—such as `WhisperSTTHandler` or `ChatCompletionsLanguageModelHandler`—and wire their input/output queues together.

This dynamic assembly means changing from Whisper to Paraformer STT requires only swapping the argument class reference, with no changes to the orchestration logic.

## Configuring Voice Settings and Audio Parameters

Voice characteristics and audio processing parameters are controlled through TTS and STT handler arguments.

### TTS Handler Configuration

The `Qwen3TTSHandlerArguments` dataclass in the arguments layer exposes parameters for voice selection, speed, and output format. When passed to `S2SPipeline`, these settings configure the `Qwen3TTSHandler` in [`src/speech_to_speech/TTS/qwen3_tts_handler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/TTS/qwen3_tts_handler.py) to generate audio with specific acoustic properties.

```python
from speech_to_speech.s2s_pipeline import S2SPipeline
from speech_to_speech.arguments_classes import Qwen3TTSHandlerArguments

# Configure voice settings at runtime

tts_args = Qwen3TTSHandlerArguments(
    qwen3_tts_model_name="Qwen/Qwen3-tts",
    # Additional voice parameters specific to the handler

)

pipeline = S2SPipeline.from_dict({
    "qwen3_tts_model_name": "Qwen/Qwen3-tts",
    "whisper_model_name": "openai/whisper-base"
})

```

### STT and VAD Configuration

Speech recognition settings are managed through `WhisperSTTHandlerArguments` and `VADHandlerArguments`. These control model selection, language detection, and voice activity detection sensitivity. The `BaseHandler` class in [`src/speech_to_speech/baseHandler.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/baseHandler.py) consumes these configurations when spawning worker threads for audio processing.

## Session Instructions and Tool Integration

The pipeline supports dynamic configuration of language model behavior through dedicated argument classes.

### LLM Handler Arguments

`LanguageModelHandlerArguments` defines session-level instructions and available tools for the `ChatCompletionsLanguageModelHandler`. This includes system prompts, maximum token limits, and tool schemas for function calling. The `SpeculativeTurnTracker` in [`src/speech_to_speech/pipeline/speculative_turns.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/pipeline/speculative_turns.py) uses these settings to optimize response latency while respecting the configured instructions.

```python

# Configure instructions and tools programmatically

config = {
    "language_model_name": "meta-llama/Meta-Llama-3-8B-Instruct",
    "system_prompt": "You are a helpful voice assistant. Use tools when necessary.",
    # Tool definitions would be passed through the appropriate argument fields

}

pipeline = S2SPipeline.from_dict(config)
pipeline.run()

```

## Runtime Configuration Patterns

You can inject configuration data at runtime through two primary mechanisms.

### Programmatic Configuration with from_dict

The `S2SPipeline.from_dict()` method accepts a dictionary mapping argument names to values, instantiating the necessary dataclasses internally. This approach is ideal for applications that need to switch configurations based on user profiles or external APIs.

```python
from speech_to_speech.s2s_pipeline import S2SPipeline

# Runtime session configuration

session_config = {
    "whisper_model_name": "openai/whisper-base",
    "qwen3_tts_model_name": "Qwen/Qwen3-tts",
    "language_model_name": "meta-llama/Meta-Llama-3-8B-Instruct",
    "use_vad": True,
}

pipeline = S2SPipeline.from_dict(session_config)
pipeline.run()

```

### Command-Line Interface

For server deployments, the pipeline uses `HfArgumentParser` to aggregate all argument classes into a unified CLI. You can pass configuration files or individual flags to override defaults defined in the dataclasses.

```bash
python -m speech_to_speech.s2s_pipeline \
    --whisper_model_name openai/whisper-base \
    --qwen3_tts_model_name Qwen/Qwen3-tts \
    --language_model_name meta-llama/Meta-Llama-3-8B-Instruct \
    --use_vad True

```

## Session Lifecycle and Control

Runtime configuration includes mechanisms for graceful session termination and state management.

### Control Messages and Session Termination

The `ControlKind` enum in [`src/speech_to_speech/pipeline/control.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/pipeline/control.py) defines `END_SESSION`, a special control message that signals all handlers to cease processing. When `BaseHandler` instances receive this message through their input queues, they shut down their worker threads cleanly, allowing you to reconfigure and restart the pipeline with new parameters without restarting the Python process.

## Monitoring Configuration via Events

The pipeline emits typed events defined in [`src/speech_to_speech/pipeline/events.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/pipeline/events.py) that allow external observers to track configuration changes and processing state.

```python
from speech_to_speech.s2s_pipeline import S2SPipeline
from speech_to_speech.pipeline.events import TranscriptionCompletedEvent, SpeechStartedEvent

pipeline = S2SPipeline.from_dict(config)

# Monitor session events

pipeline.register_event_handler(
    TranscriptionCompletedEvent, 
    lambda ev: print(f"Transcription: {ev.transcription}")
)
pipeline.register_event_handler(
    SpeechStartedEvent,
    lambda ev: print("Speech detected, processing...")
)

pipeline.run()

```

## Summary

- **Typed dataclasses** like `Qwen3TTSHandlerArguments` and `LanguageModelHandlerArguments` encapsulate configuration for each pipeline layer in `src/speech_to_speech/arguments_classes/`.
- **`S2SPipeline.from_dict()`** enables runtime configuration injection without CLI dependency, supporting dynamic voice settings and instruction updates.
- **`ParsedArguments`** aggregates all handler configurations, allowing the pipeline to construct the appropriate processing graph at initialization.
- **Control messages** such as `ControlKind.END_SESSION** manage session lifecycle, enabling clean shutdowns and configuration refreshes.
- **Event system** in [`src/speech_to_speech/pipeline/events.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/pipeline/events.py) provides observability into how runtime configuration affects processing.

## Frequently Asked Questions

### How do I change voice settings mid-session in the Hugging Face speech-to-speech pipeline?

You cannot modify TTS parameters like voice or speed while the pipeline is actively running because handlers initialize their models during `S2SPipeline.run()`. To change voice settings, send a `ControlKind.END_SESSION` message to trigger graceful shutdown, then instantiate a new `S2SPipeline` with updated `Qwen3TTSHandlerArguments` via `from_dict()`.

### What is the difference between `S2SPipeline()` and `S2SPipeline.from_dict()` for configuration?

**`S2SPipeline()`** parses arguments from `sys.argv` using `HfArgumentParser`, making it suitable for command-line execution. **`S2SPipeline.from_dict()`** accepts a Python dictionary directly, enabling programmatic configuration within applications or web servers where CLI arguments are unavailable.

### Where are session instructions stored in the argument architecture?

Session instructions reside in `LanguageModelHandlerArguments`, which configures the `ChatCompletionsLanguageModelHandler`. These arguments define the system prompt, available tools, and generation parameters that govern how the LLM responds to transcriptions during the speech-to-speech session.

### How does the pipeline handle configuration validation?

The `HfArgumentParser` validates all configuration against the type hints defined in the dataclass argument classes (e.g., `WhisperSTTHandlerArguments`) before `S2SPipeline` constructs the handler graph. Invalid types or missing required fields raise parsing errors immediately, preventing misconfigured sessions from starting.