# How to Use the Chat Completions API for LLM Backend in Speech-to-Speech

> Learn how to use the Chat Completions API for your LLM backend with huggingface speech-to-speech. Manage your LLM lifecycle easily and efficiently.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-11

---

**The Speech-to-Speech repository provides an OpenAI-compatible Chat Completions backend that handles full LLM lifecycle management through `ChatCompletionsApiModelHandler` in [`speech_to_speech/LLM/chat_completions_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/speech_to_speech/LLM/chat_completions_language_model.py).**

This backend enables you to connect any OpenAI-compatible endpoint—including vLLM, Qwen, or OpenAI itself—to your speech pipeline. The implementation wraps the standard `/v1/chat/completions` protocol with built-in streaming support, tool-call handling, and audio input capabilities.

## What Is the Chat Completions Backend

The **Chat Completions backend** is registered under the identifier `chat-completions` and lives in [`src/speech_to_speech/LLM/chat_completions_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/chat_completions_language_model.py). It inherits from `BaseOpenAICompatibleHandler` in [`base_openai_compatible_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/base_openai_compatible_language_model.py), which provides common infrastructure for OpenAI-compatible endpoints.

Key components include:

- **`ChatCompletionsApiModelHandler`** (lines 78-94): The main handler class implementing warmup, request building, and response parsing
- **`ChatCompletionsLanguageModelHandlerArguments`** (lines 9-21): Configuration dataclass exposing CLI flags like `--responses_api_base_url` and `--responses_api_stream`
- **Serialization helpers**: `_chat_messages`, `_to_chat_content_part`, `_build_chat_optional_kwargs` for converting internal `Chat` objects to OpenAI message format
- **Streaming parser**: `_iter_chat_stream_events` (lines 202-254) yielding `TextDelta`, `AssistantMessage`, `ToolCall`, and `Usage` events
- **Non-streaming parser**: `_iter_chat_response_events` (lines 56-76) for single-shot responses

The backend is registered in [`src/speech_to_speech/backend_registry.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/backend_registry.py) (lines 96-104) and instantiated through `_simple_handler_factory`.

## Configuring the Chat Completions Backend

All configuration flows through `ChatCompletionsLanguageModelHandlerArguments`, which extends `ResponsesApiLanguageModelHandlerArguments`. This design lets you reuse existing `--responses_api_*` flags across both backends.

Required parameters:

| Flag | Purpose | Default |
|------|---------|---------|
| `responses_api_base_url` | Endpoint base URL with `/v1` path | `https://api.openai.com/v1` |
| `responses_api_api_key` | Authentication key | `None` |
| `responses_api_stream` | Enable streaming responses | `True` |
| `responses_api_reasoning_effort` | Provider-specific reasoning level | `"none"` |

Set `responses_api_api_key="none"` for local servers without authentication.

## Running from Command Line

The fastest way to use the Chat Completions API for LLM backend is through the CLI:

```bash
speech-to-speech \
  --llm_backend chat-completions \
  --responses_api_base_url https://api.openai.com/v1 \
  --responses_api_api_key $OPENAI_API_KEY \
  --responses_api_stream true \
  --responses_api_reasoning_effort none \
  --tts_backend chatTTS \
  --audio_max_tokens 256

```

The `--llm_backend chat-completions` flag triggers the registry lookup in [`backend_registry.py`](https://github.com/huggingface/speech-to-speech/blob/main/backend_registry.py), which instantiates `ChatCompletionsApiModelHandler` with your arguments.

## Programmatic Usage

For custom integrations, instantiate the handler directly through the registry:

```python
from speech_to_speech.arguments_classes.chat_completions_language_model_arguments import (
    ChatCompletionsLanguageModelHandlerArguments,
)
from speech_to_speech.backend_registry import LLM_BACKENDS
from speech_to_speech.pipeline.chat import Chat

# Build configuration

args = ChatCompletionsLanguageModelHandlerArguments(
    responses_api_base_url="http://localhost:8000/v1",
    responses_api_api_key="none",  # No auth for local vLLM

    responses_api_stream=True,
    responses_api_reasoning_effort="none",
)

# Retrieve handler from registry

spec = next(s for s in LLM_BACKENDS if s.name == "chat-completions")
handler = spec.handler_factory(args)  # Returns ChatCompletionsApiModelHandler

# Prepare conversation

chat = Chat()
chat.add_user_message("What's the weather in Paris?")

# Process streaming response

for event in handler.process(chat):
    if hasattr(event, 'text'):
        print(event.text, end="", flush=True)

```

The `process` method inherited from `BaseHandler` orchestrates: serialization via `_chat_messages`, dispatch through `_request_chat_completions`, and event iteration via `_iter_chat_stream_events` or `_iter_chat_response_events`.

## Low-Level API Access

For fine-grained control, use the internal helpers directly from [`chat_completions_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/chat_completions_language_model.py):

```python
from openai import OpenAI
from speech_to_speech.LLM.chat_completions_language_model import (
    _chat_messages,
    _build_chat_optional_kwargs,
    _request_chat_completions,
)

client = OpenAI(
    api_key="sk-...",
    base_url="https://api.openai.com/v1"
)

# Serialize Chat object to OpenAI format

messages = _chat_messages(chat)
optional = _build_chat_optional_kwargs(tools=None, tool_choice=None)

# Execute request

stream = _request_chat_completions(
    client=client,
    model_name="gpt-4o-mini",
    messages=messages,
    stream=True,
    extra_body=None,
    timeout=20.0,
    optional_kwargs=optional,
)

for chunk in stream:
    content = chunk.choices[0].delta.content
    print(content or "", end="")

```

This pattern is useful when you need custom tool configurations or direct access to raw `ChatCompletionChunk` objects.

## Handling Audio Input and Tool Calls

The base handler in [`base_openai_compatible_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/base_openai_compatible_language_model.py) automatically manages:

- **Audio token limits**: Enforced via `audio_max_tokens` parameter
- **Audio type selection**: Chooses between `input_audio` and `audio_url` based on content
- **History compaction**: Prevents context window overflow
- **Speculative-turn gating**: Reduces latency for anticipated responses

Tool calls use the incremental `choices[].delta.tool_calls` protocol, with parsing implemented in `_iter_chat_stream_events` starting at line 202. Each `ToolCall` event contains `tool_name`, `arguments`, and `id` fields.

## Architecture Flow

When processing a conversation, the handler executes this sequence:

1. **Warmup**: `warmup` method primes the connection with a trivial request
2. **Serialization**: `_chat_messages` converts `Chat` to OpenAI message array
3. **Request**: `_request_chat_completions` POSTs to `/chat/completions`
4. **Parsing**: Stream or response parser yields `ProviderEvent` objects
5. **Integration**: Events flow to TTS and other downstream pipeline stages

## Summary

- The **Chat Completions backend** provides OpenAI-compatible LLM integration through `ChatCompletionsApiModelHandler` in [`src/speech_to_speech/LLM/chat_completions_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/chat_completions_language_model.py)
- Configuration uses `responses_api_*` flags via `ChatCompletionsLanguageModelHandlerArguments` for CLI and programmatic access
- **Streaming and non-streaming modes** both produce `ProviderEvent` objects through dedicated parsers in the handler
- **Tool calls and audio input** are fully supported with automatic token management by the base handler
- The backend registers as `chat-completions` in [`backend_registry.py`](https://github.com/huggingface/speech-to-speech/blob/main/backend_registry.py) and instantiates through `_simple_handler_factory`

## Frequently Asked Questions

### What providers work with the Chat Completions backend?

Any OpenAI-compatible endpoint works, including OpenAI's official API, **vLLM**, and **Qwen** deployments. The handler is specifically recommended for providers that expose the mature Chat Completions streaming specification rather than the newer Responses API.

### How do I disable streaming for the Chat Completions backend?

Set `responses_api_stream=False` in your arguments or pass `--responses_api_stream false` on the CLI. The handler automatically routes to `_iter_chat_response_events` instead of `_iter_chat_stream_events` for non-streaming operations.

### What's the difference between Chat Completions and Responses API backends?

The **Chat Completions backend** uses the established `/v1/chat/completions` endpoint with full tool-call delta support, while the Responses API targets newer provider-specific protocols. Chat Completions has broader compatibility with local inference servers like vLLM. Both share configuration infrastructure through `BaseOpenAICompatibleHandler`.

### How does audio input work with the Chat Completions handler?

Audio blobs in the `Chat` object are serialized through `_to_chat_content_part` into either `input_audio` (base64-encoded) or `audio_url` references. The base handler enforces `audio_max_tokens` limits and manages audio-type selection automatically based on content size and configuration.