Configuring Reasoning Effort and Thinking Mode Suppression for Voice Responses in Speech-to-Speech

Configure reasoning effort and suppress thinking tokens in the Hugging Face Speech-to-Speech pipeline by setting backend-specific arguments for the OpenAI Responses API LLM handler and processing output through the LMOutputProcessor handler chain.

The speech-to-speech repository implements a fully modular voice-agent architecture that routes audio through a cascade of swappable components: VAD → STT → LLM → TTS. Because the LLM stage is abstracted behind a generic BackendSpec interface, you can configure advanced generation parameters—including reasoning effort levels and thinking mode suppression—through the arguments system defined in src/speech_to_speech/arguments_classes/ and applied via the backend registry in src/speech_to_speech/backend_registry.py.

LLM Backend Configuration Architecture

The pipeline discovers and instantiates LLM handlers through a registry pattern that maps CLI flags to concrete implementations.

Backend Registration and Selection

In src/speech_to_speech/backend_registry.py, each LLM provider is registered as a BackendSpec that bundles the backend name, its argument dataclass, and a factory function. The select_backend() function resolves user selections (e.g., responses-api or chat-completions) to handler instances, while create_backend_handler() lazily loads dependencies and validates configuration.

# Conceptual flow from the registry

BackendSpec(
    name="responses-api",
    arguments_class=ResponsesAPIArguments,  # Defines reasoning_effort, etc.

    factory=create_responses_handler,
    supports_llm_proxy=True
)

This design means any parameter supported by the upstream API—such as OpenAI's reasoning_effort (low/medium/high) or thinking configuration—can be exposed by extending the appropriate arguments dataclass in src/speech_to_speech/arguments_classes/.

Argument Propagation

The run_pipeline_command() entry point in src/speech_to_speech/s2s_pipeline.py parses CLI flags into a hierarchy of dataclasses (e.g., ModuleArguments, RealtimeServerArguments, and backend-specific classes). These are wrapped in a ParsedArguments object and passed to _build_pipeline_unit(), which deep-copies configurations for each pipeline unit to ensure isolation.

When using the Responses API backend, flags prefixed with --responses_api_ map directly to the argument dataclass fields, allowing you to set reasoning parameters without modifying core pipeline code.

Implementing Reasoning Effort via Responses API Arguments

To configure reasoning effort for models that support it (such as OpenAI's o1 or o3 series), you target the responses-api backend, which communicates with OpenAI's /v1/responses endpoint.

Configuration Through CLI

When starting the server, prefix reasoning parameters with the backend name. The pipeline forwards these values to the upstream API via the LLM handler:

speech-to-speech serve \
    --llm_backend responses-api \
    --responses_api_reasoning_effort high \
    --responses_api_model o3-mini

The BackendSpec for responses-api includes supports_llm_proxy=True, enabling the RealtimeServer (defined in src/speech_to_speech/api/openai_realtime/server.py) to forward these parameters through the RealtimeService orchestrator in src/speech_to_speech/api/openai_realtime/service.py.

Programmatic Configuration

When constructing the pipeline programmatically, pass reasoning parameters through the argument list parsed by parse_arguments():

from speech_to_speech.s2s_pipeline import parse_arguments, build_pipeline
from threading import Event

args = parse_arguments(
    ["--llm_backend", "responses-api",
     "--responses_api_reasoning_effort", "medium",
     "--responses_api_thinking", "disabled"],  # Hypothetical suppression flag

    command="serve",
)

stop_event = Event()
pipeline_manager = build_pipeline(args, stop_event)
pipeline_manager.start()

The prepare_all_args() function validates these backend-specific configurations before the ThreadManager initializes the handler threads.

Suppressing Thinking Tokens in the Pipeline

Raw responses from reasoning models include thinking tokens or chain-of-thought content that should be filtered before reaching the TTS stage to avoid spoken "internal monologue."

The LMOutputProcessor Handler

In the handler chain constructed by _build_handlers() in src/speech_to_speech/s2s_pipeline.py, the LMOutputProcessor sits between the LLM and TTS stages. Its role is to transform streamed LLM output into a format suitable for speech synthesis.

To suppress thinking mode:

1. Configure the LLM backend to exclude thinking content from the API response (via thinking: {type: "disabled"} in Responses API calls)
2. Filter in the processor if the API returns thinking tokens separately—the LMOutputProcessor can strip content marked with type: "thinking" before enqueueing text for the TTS handler

The handler receives a HandlerContext containing the output queue, allowing it to intercept and sanitize content before the Qwen3TTSHandler (or other TTS backend) receives it.

WebSocket API Control

When using the OpenAI Realtime-compatible WebSocket API, the RealtimeService manages response finalization. You can configure the service to request non-reasoning responses by setting the appropriate model and parameters in the session configuration, which the service forwards to the underlying LLM handler.

Complete Configuration Examples

Server with High Reasoning Effort and Proxy Enabled

# Start server with reasoning configuration and LLM proxy for external clients

speech-to-speech serve \
    --llm_backend responses-api \
    --responses_api_reasoning_effort high \
    --responses_api_model o3-mini \
    --enable_llm_proxy

Clients can then connect via WebSocket to ws://localhost:8765/v1/realtime, and the pipeline will apply the configured reasoning effort to all voice responses.

Local macOS Deployment with MLX and Custom Reasoning Settings

speech-to-speech local --mac-optimal-settings \
    --llm_backend responses-api \
    --responses_api_reasoning_effort low

This combines Apple Silicon optimization with API-based reasoning configuration, using the MLX-accelerated audio components while delegating text generation to the remote reasoning model.

Direct LLM Proxy Usage

With --enable_llm_proxy active, you can test reasoning configuration via standard HTTP without the voice pipeline:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8765/v1", api_key="unused")
resp = client.responses.create(
    model="o3-mini",
    reasoning={"effort": "high"},
    input="Explain quantum computing briefly"
)
print(resp.output_text)

The proxy forwards the request to the upstream OpenAI API using the credentials and default parameters configured in the pipeline's arguments.

Summary

• Modular backend system: The BackendSpec registry in src/speech_to_speech/backend_registry.py allows reasoning parameters to be injected via dataclass arguments without modifying pipeline core logic.
• Responses API integration: Configure reasoning_effort and thinking mode through --responses_api_* CLI flags or programmatic arguments when using the responses-api backend.
• Content filtering: The LMOutputProcessor handler in the VAD → STT → LLM → TTS chain can suppress thinking tokens before they reach the TTS stage.
• Proxy compatibility: Enable --enable_llm_proxy to expose the configured reasoning settings through a standard OpenAI-compatible HTTP interface.

Frequently Asked Questions

How do I completely disable thinking mode in voice responses?

Configure the Responses API backend with the appropriate suppression parameter (e.g., --responses_api_thinking disabled or equivalent in your arguments dataclass). The LMOutputProcessor will then receive only the final response text, excluding internal reasoning chains, before passing it to the TTS handler.

Can I use reasoning models with local LLM backends like transformers or mlx-lm?

The analysis indicates the pipeline supports local backends (transformers, mlx-lm) and remote backends (responses-api, chat-completions). Reasoning effort configuration is specific to API-based models (OpenAI o1/o3) accessed via the responses-api backend. Local models would need to implement reasoning internally or through prompt engineering, configured via their respective argument classes in src/speech_to_speech/arguments_classes/.

Where does the pipeline handle the actual API call to OpenAI?

The LLM handler created by create_backend_handler() in src/speech_to_speech/backend_registry.py manages the connection. For the responses-api backend, the handler constructs the request using parameters from the parsed arguments dataclass and streams the response through the handler chain, with LMOutputProcessor preparing the output for the TTS stage.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}