Understanding the Difference Between responses-api and chat-completions LLM Backends for Tool-Call Streaming

The responses-api backend streams tool calls as incremental, interleaved events through the /v1/responses endpoint, while chat-completions buffers the complete tool call object before emitting it via the standard /v1/chat/completions endpoint.

The Hugging Face speech-to-speech repository enables real-time voice conversations with tool-using LLMs, offering both responses-api and chat-completions backends that handle tool-call streaming differently. Understanding these distinctions is essential for optimizing latency and choosing the right integration pattern for your OpenAI-compatible server.

Endpoint Architecture and Default Configuration

The two backends target distinct HTTP endpoints. In [src/speech_to_speech/api/openai_realtime/llm_proxy.py](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/llm_proxy.py#L30), the proxy routes requests to either /v1/responses for the responses-api backend or /v1/chat/completions for the chat-completions backend.

By default, the system selects responses-api when no backend is specified. This is defined in [src/speech_to_speech/arguments_classes/module_arguments.py](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/module_arguments.py#L36-L39), where the --llm_backend argument defaults to "responses-api".

Streaming Semantics for Tool Calls

The core difference lies in how each backend delivers tool calls during streaming.

responses-api: Incremental Event Streaming

When using responses-api, the model can emit partial tool call fragments while continuing to stream the assistant's text response. In [src/speech_to_speech/LLM/base_openai_compatible_language_model.py](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/base_openai_compatible_language_model.py#L380-L387), the _pending_tool_calls list is populated immediately as chunks arrive, allowing the system to push tool invocation events to the text_output_queue before the full response completes. This interleaved behavior enables the client to execute tools while the model is still speaking its pre-ambulatory text.

chat-completions: Buffered JSON Objects

Conversely, the chat-completions backend follows the standard OpenAI Chat Completions format. The model buffers the entire tool_calls JSON object until it is fully generated, then emits it as a single delta. The same base class parses this payload, but because the endpoint returns the tool call only when complete, the client receives the tool invocation after the surrounding text has finished streaming.

Backend Registration and Proxy Support

Both backends are registered in [src/speech_to_speech/backend_registry.py](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/backend_registry.py#L384) alongside transformers and mlx-lm options. Documentation in [src/speech_to_speech/LLM/README.md](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/README.md) outlines the four supported LLM backends and their respective handler classes.

The proxy implementation in [llm_proxy.py](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/llm_proxy.py#L156-L211) supports both backends by rewriting the model field to the locally configured name and forwarding the entire message history to the appropriate upstream endpoint, regardless of which backend is active.

Configuring the Backend via CLI

You can switch between backends using command-line flags. The --responses_api_base_url parameter accepts the base URL for the remote LLM, and the system appends the correct endpoint path based on the selected backend.

# Use responses-api (default) - streams tool calls incrementally

python -m speech_to_speech \
  --llm_backend responses-api \
  --responses_api_base_url https://api.openai.com/v1 \
  --model_name gpt-4o

# Use chat-completions - buffers full tool call objects

python -m speech_to_speech \
  --llm_backend chat-completions \
  --responses_api_base_url https://api.openai.com/v1 \
  --model_name gpt-4o

Note that when using chat-completions, the base URL should typically point to the root API endpoint (e.g., https://api.openai.com/v1), as the proxy automatically routes to /v1/chat/completions.

Summary

• responses-api targets the /v1/responses endpoint and is the default backend in the Hugging Face speech-to-speech pipeline.
• chat-completions targets the standard /v1/chat/completions endpoint used by OpenAI and compatible providers.
• Tool-call streaming: responses-api emits tool calls incrementally as they are generated, while chat-completions waits for the complete JSON object.
• Implementation: Both use BaseOpenAICompatibleLanguageModel, but the parsing logic in [base_openai_compatible_language_model.py](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/base_openai_compatible_language_model.py#L380-L387) handles the differing stream formats.
• Proxy behavior: The proxy in [llm_proxy.py](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/llm_proxy.py#L156-L211) rewrites model names and supports both backends interchangeably.

Frequently Asked Questions

What endpoint does each backend use?

The responses-api backend sends requests to /v1/responses, while chat-completions uses /v1/chat/completions. These paths are hardcoded in the proxy configuration within [llm_proxy.py](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/llm_proxy.py#L30).

Which backend should I choose for real-time tool execution?

Choose responses-api if you need low-latency tool execution that can start while the model is still generating its text response. Choose chat-completions if you are integrating with standard OpenAI-compatible services that do not support the /v1/responses contract and you prefer the traditional request-response pattern.

Can I switch backends without modifying client code?

Yes. The backend is selected server-side via the --llm_backend CLI argument. The WebSocket client interface remains identical regardless of which backend is active, as the proxy normalizes the streaming output.

How does the proxy handle model name rewriting?

When acting as a proxy, the server rewrites the incoming request's model field to match the locally configured --model_name before forwarding to the upstream provider. This behavior is implemented in [llm_proxy.py](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/llm_proxy.py#L156-L211) and applies uniformly to both responses-api and chat-completions backends.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}