`responses-api` vs `chat-completions` in Hugging Face Speech-to-Speech: Complete Backend Guide

Use --llm_backend responses-api for the older OpenAI Realtime format and --llm_backend chat-completions for modern, battle-tested tool-call streaming with better vLLM compatibility.

The Hugging Face speech-to-speech repository supports two interchangeable LLM backends that determine how your voice agent communicates with language models. Both backends target OpenAI-compatible HTTP endpoints but differ in streaming protocol, tool-call handling, and provider compatibility. This guide explains the architectural differences, configuration options, and when to choose each backend based on the actual source implementation.

Core Architectural Differences

Endpoint and Handler Classes

Each backend maps to a distinct OpenAI endpoint and dedicated handler class:

Backend	Default Endpoint	Handler Class	Source File
responses-api	POST /v1/responses	ResponsesApiModelHandler	src/speech_to_speech/LLM/responses_api_language_model.py
chat-completions	POST /v1/chat/completions	ChatCompletionsApiModelHandler	src/speech_to_speech/LLM/chat_completions_language_model.py

The --llm_backend CLI flag (parsed in module_arguments.py and injected in s2s_pipeline.py lines 104-110) selects which handler instantiates during pipeline construction.

Shared vs. Extended Arguments

Both backends inherit from ResponsesApiLanguageModelHandlerArguments (src/speech_to_speech/arguments_classes/responses_api_language_model_arguments.py), giving them identical base configuration:

• --responses_api_base_url
• --responses_api_api_key
• --responses_api_stream
• --responses_api_disable_thinking

The chat-completions backend extends these with one additional parameter in ChatCompletionsLanguageModelHandlerArguments (src/speech_to_speech/arguments_classes/chat_completions_language_model_arguments.py):

• --responses_api_reasoning_effort — forces providers to skip or reduce reasoning when they ignore the generic disable_thinking flag (relevant for vLLM issue #312)

Message Format Conversion

The responses-api backend sends chat messages as-is using the OpenAI Realtime format. The chat-completions backend performs mandatory payload adaptation in _chat_messages (lines 55-66 of chat_completions_language_model.py):

• Converts tool_calls.arguments from raw objects to JSON strings
• Rewrites multimodal parts from Realtime's input_text / input_image to Chat-Completions text / image_url shapes

This conversion ensures compatibility with providers that strictly enforce the Chat-Completions schema.

Tool-Call Streaming Implementation

Aspect	responses-api	chat-completions
Response type	Stream[ResponseChunk]	Stream[ChatCompletionChunk]
Delta location	choices[].delta.tool_calls	choices[].delta.tool_calls
Reliability	Older format, variable provider support	Mature protocol, better vLLM + Qwen compatibility

Both handlers gather tool-call deltas in _iter_stream_events and emit identical internal events (AssistantMessage, TextDelta, ToolCall, Usage), making the rest of the pipeline backend-agnostic.

CLI Configuration Examples

Responses-API Backend (Default)

speech-to-speech \
    --mode realtime \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --model_name "gpt-4o-mini" \
    --responses_api_api_key "$OPENAI_API_KEY" \
    --responses_api_stream

Chat-Completions Backend

speech-to-speech \
    --mode realtime \
    --stt parakeet-tdt \
    --llm_backend chat-completions \
    --tts qwen3 \
    --model_name "Qwen/Qwen3-4B-Instruct-2507" \
    --responses_api_base_url "http://localhost:8000/v1" \
    --responses_api_stream \
    --responses_api_reasoning_effort none

Notice that --responses_api_reasoning_effort only functions with the chat-completions backend.

Warm-Up Behavior

Each backend sends a minimal warm-up request to its respective endpoint during initialization (warmup method in chat_completions_language_model.py lines 94-102 for chat-completions; analogous implementation for responses-api).

Direct OpenAI Client Usage

Responses-API

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="any-string",
)

response = client.responses.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
)

Chat-Completions

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="any-string",
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
    extra_body={"reasoning_effort": "none"},  # optional

)

When to Choose Each Backend

Prefer responses-api when:

• You need broad compatibility with any OpenAI-compatible provider
• Your deployment uses older vLLM builds that implement the Responses spec
• You require the simplest configuration without message conversion overhead

Prefer chat-completions when:

• You're self-hosting vLLM or llama.cpp with flaky tool-call streaming via Responses (see issue #312)
• You need to suppress reasoning via responses_api_reasoning_effort on providers that ignore disable_thinking
• You're using Qwen models with tool-calling, where the Chat-Completions path has proven more reliable

Key Implementation Files

File	Purpose
src/speech_to_speech/LLM/responses_api_language_model.py	Handler for /v1/responses endpoint
src/speech_to_speech/LLM/chat_completions_language_model.py	Handler for /v1/chat/completions with _chat_messages conversion
src/speech_to_speech/arguments_classes/responses_api_language_model_arguments.py	Shared CLI arguments base class
src/speech_to_speech/arguments_classes/chat_completions_language_model_arguments.py	Extended arguments with responses_api_reasoning_effort
src/speech_to_speech/s2s_pipeline.py	Pipeline construction and backend injection (lines 104-110)

Summary

• responses-api uses POST /v1/responses with ResponsesApiModelHandler for the older OpenAI Realtime streaming format
• chat-completions uses POST /v1/chat/completions with ChatCompletionsApiModelHandler, adding message conversion and responses_api_reasoning_effort control
• Both backends share argument classes but diverge in streaming protocol reliability, particularly for tool-calling with vLLM and Qwen
• Internal events remain identical, ensuring pipeline compatibility regardless of backend choice
• Select via --llm_backend with optional --responses_api_reasoning_effort for chat-completions deployments

Frequently Asked Questions

What's the default LLM backend in speech-to-speech?

The default is responses-api. If you omit --llm_backend, the pipeline instantiates ResponsesApiModelHandler and targets POST /v1/responses. Explicitly set --llm_backend chat-completions to switch endpoints.

Why would chat-completions work when responses-api fails?

The Chat-Completions streaming protocol is more mature and widely implemented. Some vLLM versions and Qwen deployments return malformed tool-call deltas via the Responses endpoint. The chat-completions backend's stricter schema enforcement and tested _iter_stream_events implementation handle these edge cases more reliably.

Can I use different base URLs for each backend?

No—both backends use --responses_api_base_url because they share ResponsesApiLanguageModelHandlerArguments. The suffix (/responses or /chat/completions) is automatically appended by the respective handler's internal client configuration.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}