How to Choose Between `responses-api` and `chat-completions` LLM Backends in Speech-to-Speech

Use responses-api for simple OpenAI-compatible streaming, or switch to chat-completions when you need reliable tool-call streaming with vLLM/Qwen or extra control over reasoning effort.

The Hugging Face Speech-to-Speech (STS) repository offers two interchangeable LLM backends that connect your voice agent to language models through OpenAI-compatible HTTP endpoints. Both backends produce identical internal events for the downstream pipeline, but they differ in endpoint format, tool-call reliability, and configuration options. This guide explains how to select the right backend for your deployment.

---

Understanding the Two LLM Backends

STS abstracts LLM communication through two handlers that implement different OpenAI API specifications:

Backend	Default Endpoint	Handler Class	File Location
responses-api	POST /v1/responses	ResponsesApiModelHandler	src/speech_to_speech/LLM/responses_api_language_model.py
chat-completions	POST /v1/chat/completions	ChatCompletionsApiModelHandler	src/speech_to_speech/LLM/chat_completions_language_model.py

The responses-api backend uses the older Responses API format—the original streaming protocol implemented by OpenAI's Realtime API. The chat-completions backend uses the mature Chat Completions API, which offers more robust tool-call streaming and broader ecosystem compatibility.

---

Key Differences in Architecture

Endpoint and Message Format

The responses-api backend sends chat messages as-is in the OpenAI Realtime format. The chat-completions backend performs explicit conversion in its _chat_messages method to adapt payloads:

• Ensures tool_calls.arguments are JSON strings rather than raw objects
• Rewrites multimodal parts from Realtime's input_text / input_image to Chat-Completions text / image_url shapes

This conversion happens in src/speech_to_speech/LLM/chat_completions_language_model.py (lines 55-66).

Tool-Call Streaming Reliability

Tool-call deltas arrive differently in each backend:

• responses-api: Gathers deltas from Stream[ResponseChunk] with ResponseFunctionToolCall objects in choices[].delta.tool_calls
• chat-completions: Gathers deltas from Stream[ChatCompletionChunk] using the standardized Chat-Completions choices[].delta.tool_calls schema

The Chat-Completions path is more battle-tested for vLLM and Qwen model tool calling, resolving issues like GitHub issue #312 where Responses-API streaming proved unreliable.

Configuration Options

Both backends inherit from ResponsesApiLanguageModelHandlerArguments (src/speech_to_speech/arguments_classes/responses_api_language_model_arguments.py), sharing these CLI flags:

• --responses_api_base_url
• --responses_api_api_key
• --responses_api_stream
• --responses_api_disable_thinking

The chat-completions backend adds one exclusive parameter in ChatCompletionsLanguageModelHandlerArguments (src/speech_to_speech/arguments_classes/chat_completions_language_model_arguments.py):

• responses_api_reasoning_effort — sends extra_body={'reasoning_effort': <value>} to force providers to skip or reduce reasoning when the generic disable_thinking flag is ignored

Warm-Up Behavior

Each backend validates connectivity through its respective endpoint during pipeline initialization:

• responses-api: warmup() sends to /v1/responses
• chat-completions: warmup() sends to /v1/chat/completions (lines 94-102 of chat_completions_language_model.py)

---

When to Use Each Backend

Choose responses-api When...

• You need simplest compatibility with any OpenAI-compatible provider
• Your deployment uses older vLLM builds that lack Chat-Completions tool-call fixes
• You prefer the original streaming format without message conversion overhead

Choose chat-completions When...

• You run self-hosted vLLM or llama.cpp with flaky Responses-API tool-call streaming
• You use Qwen models with tool calling (addresses issue #312)
• You need responses_api_reasoning_effort to suppress reasoning on providers that ignore disable_thinking

---

Code Examples

CLI: Responses-API Backend (Default)

speech-to-speech \
    --mode realtime \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --model_name "gpt-4o-mini" \
    --responses_api_api_key "$OPENAI_API_KEY" \
    --responses_api_stream

CLI: Chat-Completions Backend

speech-to-speech \
    --mode realtime \
    --stt parakeet-tdt \
    --llm_backend chat-completions \
    --tts qwen3 \
    --model_name "Qwen/Qwen3-4B-Instruct-2507" \
    --responses_api_base_url "http://localhost:8000/v1" \
    --responses_api_stream \
    --responses_api_reasoning_effort none

Python: Direct OpenAI Client (Responses-API)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="any-string",
)

response = client.responses.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
)

Python: Direct Openai Client (Chat-Completions)

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="any-string",
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
    extra_body={"reasoning_effort": "none"},
)

---

How Backend Selection Works in the Pipeline

The --llm_backend flag is parsed in module_arguments.py and injected during pipeline construction in src/speech_to_speech/s2s_pipeline.py (lines 104-110). Regardless of your choice, both handlers emit identical internal events:

• AssistantMessage
• TextDelta
• ToolCall
• Usage

This means the rest of your pipeline—VAD, STT, and TTS—remains completely agnostic to which LLM backend you select.

---

Summary

• responses-api uses /v1/responses with the older streaming format; simplest for generic OpenAI compatibility
• chat-completions uses /v1/chat/completions with robust tool-call streaming; preferred for vLLM/Qwen deployments
• Both share argument classes from ResponsesApiLanguageModelHandlerArguments; chat-completions adds responses_api_reasoning_effort
• Backend selection happens via --llm_backend with zero impact on downstream pipeline components
• Internal message conversion in chat_completions_language_model.py adapts Realtime format to Chat-Completions schema

---

Frequently Asked Questions

What is the default LLM backend in Speech-to-Speech?

The default is responses-api. You only need to specify --llm_backend explicitly when switching to chat-completions.

Why does chat-completions have responses_api_ prefixed arguments?

Both backends inherit from ResponsesApiLanguageModelHandlerArguments for backward compatibility. The prefix reflects this shared ancestry rather than endpoint-specific naming.

Can I switch backends without changing my model server?

Yes, provided your server implements both endpoints. Most OpenAI-compatible servers expose /v1/chat/completions; /v1/responses is less common outside dedicated Realtime API implementations.

When should I use responses_api_reasoning_effort?

Use it when your provider ignores the generic --disable_thinking flag and you need to suppress reasoning tokens. This occurs with certain vLLM configurations where reasoning effort must be explicitly set in extra_body rather than through standard parameters.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}