# Understanding the Difference Between responses-api and chat-completions LLM Backends for Tool-Call Streaming

> Understand responses-api vs chat-completions LLM backends for tool call streaming. Learn how each streams tool calls differently for efficient integration.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: deep-dive
- Published: 2026-08-08

---

**The `responses-api` backend streams tool calls as incremental, interleaved events through the `/v1/responses` endpoint, while `chat-completions` buffers the complete tool call object before emitting it via the standard `/v1/chat/completions` endpoint.**

The Hugging Face `speech-to-speech` repository enables real-time voice conversations with tool-using LLMs, offering both `responses-api` and `chat-completions` backends that handle tool-call streaming differently. Understanding these distinctions is essential for optimizing latency and choosing the right integration pattern for your OpenAI-compatible server.

## Endpoint Architecture and Default Configuration

The two backends target distinct HTTP endpoints. In [[`src/speech_to_speech/api/openai_realtime/llm_proxy.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/llm_proxy.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/llm_proxy.py#L30), the proxy routes requests to either `/v1/responses` for the `responses-api` backend or `/v1/chat/completions` for the `chat-completions` backend.

By default, the system selects `responses-api` when no backend is specified. This is defined in [[`src/speech_to_speech/arguments_classes/module_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/module_arguments.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/module_arguments.py#L36-L39), where the `--llm_backend` argument defaults to `"responses-api"`.

## Streaming Semantics for Tool Calls

The core difference lies in how each backend delivers tool calls during streaming.

### responses-api: Incremental Event Streaming

When using `responses-api`, the model can emit **partial tool call fragments** while continuing to stream the assistant's text response. In [[`src/speech_to_speech/LLM/base_openai_compatible_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/base_openai_compatible_language_model.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/base_openai_compatible_language_model.py#L380-L387), the `_pending_tool_calls` list is populated immediately as chunks arrive, allowing the system to push tool invocation events to the `text_output_queue` before the full response completes. This interleaved behavior enables the client to execute tools while the model is still speaking its pre-ambulatory text.

### chat-completions: Buffered JSON Objects

Conversely, the `chat-completions` backend follows the standard OpenAI Chat Completions format. The model buffers the entire `tool_calls` JSON object until it is fully generated, then emits it as a single delta. The same base class parses this payload, but because the endpoint returns the tool call only when complete, the client receives the tool invocation **after** the surrounding text has finished streaming.

## Backend Registration and Proxy Support

Both backends are registered in [[`src/speech_to_speech/backend_registry.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/backend_registry.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/backend_registry.py#L384) alongside `transformers` and `mlx-lm` options. Documentation in [[`src/speech_to_speech/LLM/README.md`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/README.md)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/README.md) outlines the four supported LLM backends and their respective handler classes.

The proxy implementation in [[`llm_proxy.py`](https://github.com/huggingface/speech-to-speech/blob/main/llm_proxy.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/llm_proxy.py#L156-L211) supports both backends by rewriting the `model` field to the locally configured name and forwarding the entire message history to the appropriate upstream endpoint, regardless of which backend is active.

## Configuring the Backend via CLI

You can switch between backends using command-line flags. The `--responses_api_base_url` parameter accepts the base URL for the remote LLM, and the system appends the correct endpoint path based on the selected backend.

```bash

# Use responses-api (default) - streams tool calls incrementally

python -m speech_to_speech \
  --llm_backend responses-api \
  --responses_api_base_url https://api.openai.com/v1 \
  --model_name gpt-4o

```

```bash

# Use chat-completions - buffers full tool call objects

python -m speech_to_speech \
  --llm_backend chat-completions \
  --responses_api_base_url https://api.openai.com/v1 \
  --model_name gpt-4o

```

Note that when using `chat-completions`, the base URL should typically point to the root API endpoint (e.g., `https://api.openai.com/v1`), as the proxy automatically routes to `/v1/chat/completions`.

## Summary

- **`responses-api`** targets the `/v1/responses` endpoint and is the **default** backend in the Hugging Face `speech-to-speech` pipeline.
- **`chat-completions`** targets the standard `/v1/chat/completions` endpoint used by OpenAI and compatible providers.
- **Tool-call streaming**: `responses-api` emits tool calls incrementally as they are generated, while `chat-completions` waits for the complete JSON object.
- **Implementation**: Both use `BaseOpenAICompatibleLanguageModel`, but the parsing logic in [[`base_openai_compatible_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/base_openai_compatible_language_model.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/base_openai_compatible_language_model.py#L380-L387) handles the differing stream formats.
- **Proxy behavior**: The proxy in [[`llm_proxy.py`](https://github.com/huggingface/speech-to-speech/blob/main/llm_proxy.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/llm_proxy.py#L156-L211) rewrites model names and supports both backends interchangeably.

## Frequently Asked Questions

### What endpoint does each backend use?

The `responses-api` backend sends requests to `/v1/responses`, while `chat-completions` uses `/v1/chat/completions`. These paths are hardcoded in the proxy configuration within [[`llm_proxy.py`](https://github.com/huggingface/speech-to-speech/blob/main/llm_proxy.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/llm_proxy.py#L30).

### Which backend should I choose for real-time tool execution?

Choose `responses-api` if you need **low-latency tool execution** that can start while the model is still generating its text response. Choose `chat-completions` if you are integrating with standard OpenAI-compatible services that do not support the `/v1/responses` contract and you prefer the traditional request-response pattern.

### Can I switch backends without modifying client code?

Yes. The backend is selected server-side via the `--llm_backend` CLI argument. The WebSocket client interface remains identical regardless of which backend is active, as the proxy normalizes the streaming output.

### How does the proxy handle model name rewriting?

When acting as a proxy, the server rewrites the incoming request's `model` field to match the locally configured `--model_name` before forwarding to the upstream provider. This behavior is implemented in [[`llm_proxy.py`](https://github.com/huggingface/speech-to-speech/blob/main/llm_proxy.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/api/openai_realtime/llm_proxy.py#L156-L211) and applies uniformly to both `responses-api` and `chat-completions` backends.