How to Configure Generation Parameters in Huggingface Speech-to-Speech: Temperature and Max Tokens

Set llm_gen_temperature and audio_max_tokens via the argument dataclasses when constructing your S2SPipeline, and the library automatically injects these values into LLM requests using kwargs.setdefault().

The Hugging Face speech-to-speech library exposes all generation-time settings through structured argument classes and the language model base classes. Whether you need to control response randomness or limit output length, these parameters flow from dataclass fields through to the actual API calls in a transparent, testable way.

Understanding the Core Parameters

Temperature

Temperature controls the randomness of LLM output. In the speech-to-speech codebase, this value is stored as audio_temperature (default 0.0) in BaseOpenAICompatibleLanguageModel and injected into request payloads via:

kwargs.setdefault("temperature", self.audio_temperature)

This pattern appears in both responses_api_language_model.py and base_openai_compatible_language_model.py.

Max Tokens

Max tokens (also referred to as max_new_tokens) caps the number of tokens the model may generate. The default 256 is defined as audio_max_tokens in base_openai_compatible_language_model.py and applied with:

kwargs.setdefault("max_tokens", max_tokens or self.audio_max_tokens)

Available Configuration Fields

Parameter	Dataclass Field	Default	Usage
llm_gen_temperature	LanguageModelArguments.llm_gen_temperature	0.0	General LLM backend
responses_api_audio_temperature	ResponsesAPILanguageModelArguments.responses_api_audio_temperature	0.0	Responses API backend
audio_max_tokens	BaseOpenAICompatibleLanguageModel.audio_max_tokens	256	OpenAI-compatible backends
responses_api_audio_max_tokens	ResponsesAPILanguageModelArguments.responses_api_audio_max_tokens	256	Responses API backend
pocket_tts_max_tokens	PocketTTSArguments.pocket_tts_max_tokens	50	Pocket-TTS handler

Configuring Generation Parameters in Your Pipeline

Method 1: Explicit Argument Objects

The most explicit approach creates dedicated argument instances and passes them to S2SPipeline:

from speech_to_speech.s2s_pipeline import S2SPipeline
from speech_to_speech.arguments_classes.language_model_arguments import LanguageModelArguments
from speech_to_speech.arguments_classes.responses_api_language_model_arguments import (
    ResponsesAPILanguageModelArguments,
)

# Configure generation parameters

llm_args = LanguageModelArguments(
    llm_gen_temperature=0.7,
)

responses_api_args = ResponsesAPILanguageModelArguments(
    responses_api_audio_temperature=0.7,
    responses_api_audio_max_tokens=150,
)

# Pass to pipeline

pipeline = S2SPipeline(
    language_model_args=llm_args,
    responses_api_language_model_args=responses_api_args,
)

output = pipeline.run_text_prompt("Tell me a short joke.")

Method 2: Direct Keyword Arguments

For quick configuration, pass values directly as keywords:

pipeline = S2SPipeline(
    llm_gen_temperature=0.9,
    responses_api_audio_max_tokens=200,
)

The pipeline accepts these through **kwargs and routes them to the appropriate backend handlers.

Where These Parameters Flow Through

BaseOpenAICompatibleLanguageModel

In LLM/base_openai_compatible_language_model.py, the constructor captures your settings:

self.audio_temperature = temperature
self.audio_max_tokens = max_tokens

Later, during the actual API call, the class injects defaults only when not already present:

kwargs.setdefault("temperature", self.audio_temperature)
kwargs.setdefault("max_tokens", max_tokens or self.audio_max_tokens)

ResponsesAPILanguageModel

The ResponsesAPILanguageModel handler in LLM/responses_api_language_model.py forwards stored values directly when constructing model requests, accessing self.audio_temperature and self.audio_max_tokens from its parent class.

Verifying Your Configuration

The test suite confirms this wiring works correctly. In tests/test_responses_api_language_model.py, assertions verify that setting handler.audio_temperature = 0.0 results in payload values of "temperature": 0.0. Similarly, tests/test_chat_completions_backend.py validates max_tokens propagation through the request chain.

Key Source Files

• arguments_classes/language_model_arguments.py — defines llm_gen_temperature
• arguments_classes/responses_api_language_model_arguments.py — holds Responses API-specific fields
• LLM/base_openai_compatible_language_model.py — stores and injects temperature and max tokens
• LLM/responses_api_language_model.py — forwards values to the model API
• tests/test_responses_api_language_model.py — validates temperature in payloads
• tests/test_chat_completions_backend.py — validates max tokens handling

Summary

• Temperature and max tokens are configured through dataclass fields in LanguageModelArguments and ResponsesAPILanguageModelArguments
• Default values are 0.0 for temperature and 256 for max tokens across most backends
• The BaseOpenAICompatibleLanguageModel stores these values and injects them via kwargs.setdefault() during API calls
• Both explicit argument objects and direct keyword arguments work when constructing S2SPipeline
• Test coverage in test_responses_api_language_model.py and test_chat_completions_backend.py guarantees correct parameter propagation

Frequently Asked Questions

What happens if I don't specify generation parameters?

The pipeline uses conservative defaults: 0.0 temperature (deterministic output) and 256 max tokens. These are defined in base_openai_compatible_language_model.py and apply automatically to all OpenAI-compatible backends.

Can I override parameters for a single request?

The current architecture applies configuration at pipeline construction time. For per-request overrides, instantiate a new S2SPipeline with modified arguments or modify the handler's audio_temperature and audio_max_tokens attributes directly before calling the model.

Why are there separate fields for Responses API and general LLM backends?

The Responses API (responses_api_audio_temperature, responses_api_audio_max_tokens) supports audio-native models with distinct tokenization and pricing characteristics. The general LLM fields (llm_gen_temperature, audio_max_tokens) serve text-based backends. This separation allows independent tuning when mixing backend types in advanced pipelines.

How does temperature affect speech-to-speech output?

Lower temperatures (closer to 0.0) produce more consistent, predictable responses—useful for structured tasks. Higher values (e.g., 0.7–0.9) increase variability and creativity in generated speech content. The speech-to-speech default of 0.0 prioritizes reliability over spontaneity.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}