# How to Configure Generation Parameters in Huggingface Speech-to-Speech: Temperature and Max Tokens

> Master Huggingface Speech-to-Speech generation parameters like temperature and max tokens. Learn to configure llm_gen_temperature and audio_max_tokens for optimal results in your S2SPipeline.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-06

---

**Set `llm_gen_temperature` and `audio_max_tokens` via the argument dataclasses when constructing your `S2SPipeline`, and the library automatically injects these values into LLM requests using `kwargs.setdefault()`.**

The Hugging Face **speech-to-speech** library exposes all generation-time settings through structured argument classes and the language model base classes. Whether you need to control response randomness or limit output length, these parameters flow from dataclass fields through to the actual API calls in a transparent, testable way.

## Understanding the Core Parameters

### Temperature

**Temperature** controls the randomness of LLM output. In the speech-to-speech codebase, this value is stored as `audio_temperature` (default `0.0`) in `BaseOpenAICompatibleLanguageModel` and injected into request payloads via:

```python
kwargs.setdefault("temperature", self.audio_temperature)

```

This pattern appears in both [`responses_api_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/responses_api_language_model.py) and [`base_openai_compatible_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/base_openai_compatible_language_model.py).

### Max Tokens

**Max tokens** (also referred to as `max_new_tokens`) caps the number of tokens the model may generate. The default `256` is defined as `audio_max_tokens` in [`base_openai_compatible_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/base_openai_compatible_language_model.py) and applied with:

```python
kwargs.setdefault("max_tokens", max_tokens or self.audio_max_tokens)

```

## Available Configuration Fields

| Parameter | Dataclass Field | Default | Usage |
|-----------|-----------------|---------|-------|
| `llm_gen_temperature` | `LanguageModelArguments.llm_gen_temperature` | `0.0` | General LLM backend |
| `responses_api_audio_temperature` | `ResponsesAPILanguageModelArguments.responses_api_audio_temperature` | `0.0` | Responses API backend |
| `audio_max_tokens` | `BaseOpenAICompatibleLanguageModel.audio_max_tokens` | `256` | OpenAI-compatible backends |
| `responses_api_audio_max_tokens` | `ResponsesAPILanguageModelArguments.responses_api_audio_max_tokens` | `256` | Responses API backend |
| `pocket_tts_max_tokens` | `PocketTTSArguments.pocket_tts_max_tokens` | `50` | Pocket-TTS handler |

## Configuring Generation Parameters in Your Pipeline

### Method 1: Explicit Argument Objects

The most explicit approach creates dedicated argument instances and passes them to `S2SPipeline`:

```python
from speech_to_speech.s2s_pipeline import S2SPipeline
from speech_to_speech.arguments_classes.language_model_arguments import LanguageModelArguments
from speech_to_speech.arguments_classes.responses_api_language_model_arguments import (
    ResponsesAPILanguageModelArguments,
)

# Configure generation parameters

llm_args = LanguageModelArguments(
    llm_gen_temperature=0.7,
)

responses_api_args = ResponsesAPILanguageModelArguments(
    responses_api_audio_temperature=0.7,
    responses_api_audio_max_tokens=150,
)

# Pass to pipeline

pipeline = S2SPipeline(
    language_model_args=llm_args,
    responses_api_language_model_args=responses_api_args,
)

output = pipeline.run_text_prompt("Tell me a short joke.")

```

### Method 2: Direct Keyword Arguments

For quick configuration, pass values directly as keywords:

```python
pipeline = S2SPipeline(
    llm_gen_temperature=0.9,
    responses_api_audio_max_tokens=200,
)

```

The pipeline accepts these through `**kwargs` and routes them to the appropriate backend handlers.

## Where These Parameters Flow Through

### BaseOpenAICompatibleLanguageModel

In [`LLM/base_openai_compatible_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/LLM/base_openai_compatible_language_model.py), the constructor captures your settings:

```python
self.audio_temperature = temperature
self.audio_max_tokens = max_tokens

```

Later, during the actual API call, the class injects defaults only when not already present:

```python
kwargs.setdefault("temperature", self.audio_temperature)
kwargs.setdefault("max_tokens", max_tokens or self.audio_max_tokens)

```

### ResponsesAPILanguageModel

The `ResponsesAPILanguageModel` handler in [`LLM/responses_api_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/LLM/responses_api_language_model.py) forwards stored values directly when constructing model requests, accessing `self.audio_temperature` and `self.audio_max_tokens` from its parent class.

## Verifying Your Configuration

The test suite confirms this wiring works correctly. In [`tests/test_responses_api_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/tests/test_responses_api_language_model.py), assertions verify that setting `handler.audio_temperature = 0.0` results in payload values of `"temperature": 0.0`. Similarly, [`tests/test_chat_completions_backend.py`](https://github.com/huggingface/speech-to-speech/blob/main/tests/test_chat_completions_backend.py) validates `max_tokens` propagation through the request chain.

## Key Source Files

- [`arguments_classes/language_model_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/arguments_classes/language_model_arguments.py) — defines `llm_gen_temperature`
- [`arguments_classes/responses_api_language_model_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/arguments_classes/responses_api_language_model_arguments.py) — holds Responses API-specific fields
- [`LLM/base_openai_compatible_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/LLM/base_openai_compatible_language_model.py) — stores and injects temperature and max tokens
- [`LLM/responses_api_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/LLM/responses_api_language_model.py) — forwards values to the model API
- [`tests/test_responses_api_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/tests/test_responses_api_language_model.py) — validates temperature in payloads
- [`tests/test_chat_completions_backend.py`](https://github.com/huggingface/speech-to-speech/blob/main/tests/test_chat_completions_backend.py) — validates max tokens handling

## Summary

- **Temperature** and **max tokens** are configured through dataclass fields in `LanguageModelArguments` and `ResponsesAPILanguageModelArguments`
- Default values are `0.0` for temperature and `256` for max tokens across most backends
- The `BaseOpenAICompatibleLanguageModel` stores these values and injects them via `kwargs.setdefault()` during API calls
- Both explicit argument objects and direct keyword arguments work when constructing `S2SPipeline`
- Test coverage in [`test_responses_api_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/test_responses_api_language_model.py) and [`test_chat_completions_backend.py`](https://github.com/huggingface/speech-to-speech/blob/main/test_chat_completions_backend.py) guarantees correct parameter propagation

## Frequently Asked Questions

### What happens if I don't specify generation parameters?

The pipeline uses conservative defaults: `0.0` temperature (deterministic output) and `256` max tokens. These are defined in [`base_openai_compatible_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/base_openai_compatible_language_model.py) and apply automatically to all OpenAI-compatible backends.

### Can I override parameters for a single request?

The current architecture applies configuration at pipeline construction time. For per-request overrides, instantiate a new `S2SPipeline` with modified arguments or modify the handler's `audio_temperature` and `audio_max_tokens` attributes directly before calling the model.

### Why are there separate fields for Responses API and general LLM backends?

The Responses API (`responses_api_audio_temperature`, `responses_api_audio_max_tokens`) supports audio-native models with distinct tokenization and pricing characteristics. The general LLM fields (`llm_gen_temperature`, `audio_max_tokens`) serve text-based backends. This separation allows independent tuning when mixing backend types in advanced pipelines.

### How does temperature affect speech-to-speech output?

Lower temperatures (closer to `0.0`) produce more consistent, predictable responses—useful for structured tasks. Higher values (e.g., `0.7`–`0.9`) increase variability and creativity in generated speech content. The speech-to-speech default of `0.0` prioritizes reliability over spontaneity.