# How to Choose Between `responses-api` and `chat-completions` LLM Backends in Speech-to-Speech

> Choose responses-api for simple OpenAI streaming or chat-completions for reliable tool-call streaming with vLLM/Qwen and advanced reasoning control.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: best-practices
- Published: 2026-08-06

---

**Use `responses-api` for simple OpenAI-compatible streaming, or switch to `chat-completions` when you need reliable tool-call streaming with vLLM/Qwen or extra control over reasoning effort.**

The **Hugging Face Speech-to-Speech (STS)** repository offers two interchangeable LLM backends that connect your voice agent to language models through OpenAI-compatible HTTP endpoints. Both backends produce identical internal events for the downstream pipeline, but they differ in endpoint format, tool-call reliability, and configuration options. This guide explains how to select the right backend for your deployment.

---

## Understanding the Two LLM Backends

STS abstracts LLM communication through two handlers that implement different OpenAI API specifications:

| Backend | Default Endpoint | Handler Class | File Location |
|---------|------------------|---------------|---------------|
| **`responses-api`** | `POST /v1/responses` | `ResponsesApiModelHandler` | [`src/speech_to_speech/LLM/responses_api_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/responses_api_language_model.py) |
| **`chat-completions`** | `POST /v1/chat/completions` | `ChatCompletionsApiModelHandler` | [`src/speech_to_speech/LLM/chat_completions_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/chat_completions_language_model.py) |

The **`responses-api`** backend uses the older **Responses API** format—the original streaming protocol implemented by OpenAI's Realtime API. The **`chat-completions`** backend uses the mature **Chat Completions API**, which offers more robust tool-call streaming and broader ecosystem compatibility.

---

## Key Differences in Architecture

### Endpoint and Message Format

The `responses-api` backend sends chat messages as-is in the OpenAI Realtime format. The `chat-completions` backend performs explicit conversion in its `_chat_messages` method to adapt payloads:

- Ensures `tool_calls.arguments` are JSON strings rather than raw objects
- Rewrites multimodal parts from Realtime's `input_text` / `input_image` to Chat-Completions `text` / `image_url` shapes

This conversion happens in [`src/speech_to_speech/LLM/chat_completions_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/chat_completions_language_model.py) (lines 55-66).

### Tool-Call Streaming Reliability

Tool-call deltas arrive differently in each backend:

- **`responses-api`**: Gathers deltas from `Stream[ResponseChunk]` with `ResponseFunctionToolCall` objects in `choices[].delta.tool_calls`
- **`chat-completions`**: Gathers deltas from `Stream[ChatCompletionChunk]` using the standardized Chat-Completions `choices[].delta.tool_calls` schema

The Chat-Completions path is more battle-tested for **vLLM** and **Qwen** model tool calling, resolving issues like GitHub issue #312 where Responses-API streaming proved unreliable.

### Configuration Options

Both backends inherit from **`ResponsesApiLanguageModelHandlerArguments`** ([`src/speech_to_speech/arguments_classes/responses_api_language_model_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/responses_api_language_model_arguments.py)), sharing these CLI flags:

- `--responses_api_base_url`
- `--responses_api_api_key`
- `--responses_api_stream`
- `--responses_api_disable_thinking`

The `chat-completions` backend adds one exclusive parameter in **`ChatCompletionsLanguageModelHandlerArguments`** ([`src/speech_to_speech/arguments_classes/chat_completions_language_model_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/chat_completions_language_model_arguments.py)):

- **`responses_api_reasoning_effort`** — sends `extra_body={'reasoning_effort': <value>}` to force providers to skip or reduce reasoning when the generic `disable_thinking` flag is ignored

### Warm-Up Behavior

Each backend validates connectivity through its respective endpoint during pipeline initialization:

- `responses-api`: `warmup()` sends to `/v1/responses`
- `chat-completions`: `warmup()` sends to `/v1/chat/completions` (lines 94-102 of [`chat_completions_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/chat_completions_language_model.py))

---

## When to Use Each Backend

### Choose `responses-api` When...

- You need **simplest compatibility** with any OpenAI-compatible provider
- Your deployment uses **older vLLM builds** that lack Chat-Completions tool-call fixes
- You prefer the **original streaming format** without message conversion overhead

### Choose `chat-completions` When...

- You run **self-hosted vLLM or llama.cpp** with flaky Responses-API tool-call streaming
- You use **Qwen models** with tool calling (addresses issue #312)
- You need **`responses_api_reasoning_effort`** to suppress reasoning on providers that ignore `disable_thinking`

---

## Code Examples

### CLI: Responses-API Backend (Default)

```bash
speech-to-speech \
    --mode realtime \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --tts qwen3 \
    --model_name "gpt-4o-mini" \
    --responses_api_api_key "$OPENAI_API_KEY" \
    --responses_api_stream

```

### CLI: Chat-Completions Backend

```bash
speech-to-speech \
    --mode realtime \
    --stt parakeet-tdt \
    --llm_backend chat-completions \
    --tts qwen3 \
    --model_name "Qwen/Qwen3-4B-Instruct-2507" \
    --responses_api_base_url "http://localhost:8000/v1" \
    --responses_api_stream \
    --responses_api_reasoning_effort none

```

### Python: Direct OpenAI Client (Responses-API)

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="any-string",
)

response = client.responses.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
)

```

### Python: Direct Openai Client (Chat-Completions)

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="any-string",
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
    stream=True,
    extra_body={"reasoning_effort": "none"},
)

```

---

## How Backend Selection Works in the Pipeline

The `--llm_backend` flag is parsed in [`module_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/module_arguments.py) and injected during pipeline construction in [`src/speech_to_speech/s2s_pipeline.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/s2s_pipeline.py) (lines 104-110). Regardless of your choice, both handlers emit identical internal events:

- `AssistantMessage`
- `TextDelta`
- `ToolCall`
- `Usage`

This means the **rest of your pipeline—VAD, STT, and TTS—remains completely agnostic** to which LLM backend you select.

---

## Summary

- **`responses-api`** uses `/v1/responses` with the older streaming format; simplest for generic OpenAI compatibility
- **`chat-completions`** uses `/v1/chat/completions` with robust tool-call streaming; preferred for vLLM/Qwen deployments
- Both share argument classes from `ResponsesApiLanguageModelHandlerArguments`; `chat-completions` adds `responses_api_reasoning_effort`
- Backend selection happens via `--llm_backend` with zero impact on downstream pipeline components
- Internal message conversion in [`chat_completions_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/chat_completions_language_model.py) adapts Realtime format to Chat-Completions schema

---

## Frequently Asked Questions

### What is the default LLM backend in Speech-to-Speech?

The default is **`responses-api`**. You only need to specify `--llm_backend` explicitly when switching to `chat-completions`.

### Why does `chat-completions` have `responses_api_` prefixed arguments?

Both backends inherit from `ResponsesApiLanguageModelHandlerArguments` for backward compatibility. The prefix reflects this shared ancestry rather than endpoint-specific naming.

### Can I switch backends without changing my model server?

Yes, provided your server implements both endpoints. Most OpenAI-compatible servers expose `/v1/chat/completions`; `/v1/responses` is less common outside dedicated Realtime API implementations.

### When should I use `responses_api_reasoning_effort`?

Use it when your provider ignores the generic `--disable_thinking` flag and you need to suppress reasoning tokens. This occurs with certain vLLM configurations where reasoning effort must be explicitly set in `extra_body` rather than through standard parameters.