# How to Use vLLM with the Responses API Backend for Speech-to-Speech Pipelines

> Learn how to use vLLM with the Responses API backend for speech-to-speech pipelines. Point the speech-to-speech library to your vLLM HTTP endpoint using responses_api_base_url and responses_api_model_name.

- Repository: [Hugging Face/speech-to-speech](https://github.com/huggingface/speech-to-speech)
- Tags: how-to-guide
- Published: 2026-08-11

---

**Point the speech-to-speech library to a locally hosted vLLM server by setting `--responses_api_base_url` to your vLLM HTTP endpoint and matching the `--responses_api_model_name` to the value served by vLLM.**

The Hugging Face `speech-to-speech` library orchestrates a modular voice agent pipeline (VAD → STT → LLM → TTS) where the LLM component communicates via any OpenAI-compatible **Responses API** server. Running this on a local **vLLM** instance requires only endpoint configuration—no code changes needed. This guide walks through CLI setup, Python API usage, and the alternative Chat Completions backend for tool-call reliability.

---

## Start a vLLM Server with OpenAI-Compatible Responses API

Before connecting the pipeline, launch vLLM with the Responses API enabled. The server must expose the `/v1/responses` endpoint that the library expects.

```bash
vllm serve Qwen/Qwen-7B-Chat \
    --port 8000 \
    --api-key dummy \
    --served-model-name qwen

```

Key flags explained:

- `--served-model-name qwen` — registers the model under this identifier in the OpenAI-compatible API
- `--api-key dummy` — vLLM requires a key even for local use; match this value in pipeline configuration

The server now accepts requests at `http://localhost:8000/v1`.

---

## Configure the Speech-to-Speech Pipeline for vLLM

The library's CLI arguments for the Responses API backend are defined in [[`src/speech_to_speech/arguments_classes/responses_api_language_model_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/responses_api_language_model_arguments.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/arguments_classes/responses_api_language_model_arguments.py#L16-L53) (lines 16–53). These map directly to vLLM connection parameters.

### CLI Configuration

```bash
speech_to_speech \
    --responses_api_base_url http://localhost:8000/v1 \
    --responses_api_api_key dummy \
    --responses_api_model_name qwen \
    --responses_api_stream true \
    --responses_api_audio_content_type input_audio

```

| Argument | Purpose | vLLM Mapping |
|----------|---------|--------------|
| `--responses_api_base_url` | Root URL for the Responses API | `http://localhost:8000/v1` |
| `--responses_api_api_key` | Authentication token | Must match `--api-key` from vLLM launch |
| `--responses_api_model_name` | Model identifier to request | Must match `--served-model-name` |
| `--responses_api_stream` | Enable server-sent events for responses | Recommended for real-time pipelines |
| `--responses_api_audio_content_type` | Audio input format for multimodal models | `input_audio` for audio-native LLMs |

The `--responses_api_base_url` parameter is documented in the repository README at [line 382](https://github.com/huggingface/speech-to-speech/blob/main/README.md#L382), which describes self-hosted server integration.

---

## Python API Usage for vLLM Integration

For programmatic control, instantiate `ResponsesApiLanguageModelHandlerArguments` and pass it to the pipeline constructor.

```python
from speech_to_speech import SpeechToSpeechPipeline
from speech_to_speech.arguments_classes.responses_api_language_model_arguments import (
    ResponsesApiLanguageModelHandlerArguments,
)

llm_args = ResponsesApiLanguageModelHandlerArguments(
    responses_api_base_url="http://localhost:8000/v1",
    responses_api_api_key="dummy",
    model_name="qwen",
    responses_api_stream=True,
    responses_api_audio_content_type="input_audio",
)

pipeline = SpeechToSpeechPipeline(
    llm_handler="responses_api",   # selects ResponsesApiModelHandler

    llm_handler_kwargs=llm_args.to_dict(),
)
pipeline.run()

```

The `llm_handler="responses_api"` string maps to `ResponsesApiModelHandler` via the backend registry in [[`src/speech_to_speech/backend_registry.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/backend_registry.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/backend_registry.py#L389-L393) (lines 389–393). The actual HTTP client implementation lives in [[`src/speech_to_speech/LLM/responses_api_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/responses_api_language_model.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/responses_api_language_model.py#L43-L50) (lines 43–50), which formats requests to the `/v1/responses` endpoint.

---

## Tool-Call Streaming: When to Use Chat Completions Instead

The Responses API streaming implementation in some vLLM builds exhibits reliability issues with tool calls. The library maintainers recommend the **Chat Completions** backend for production tool-use scenarios, as noted in the README at [line 431](https://github.com/huggingface/speech-to-speech/blob/main/README.md#L431).

### Switching to Chat Completions

```bash
speech_to_speech \
    --chat_completions_base_url http://localhost:8000/v1 \
    --chat_completions_model_name qwen \
    --chat_completions_stream true

```

This selects `ChatCompletionsApiModelHandler` from [[`src/speech_to_speech/LLM/chat_completions_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/chat_completions_language_model.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/chat_completions_language_model.py#L278-L285) (lines 278–285). The handler structure mirrors the Responses API implementation—see the parallel initialization code at [line 283](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/LLM/chat_completions_language_model.py#L283)—ensuring identical behavior for non-tool-call use cases.

Both handlers are registered in [[`backend_registry.py`](https://github.com/huggingface/speech-to-speech/blob/main/backend_registry.py)](https://github.com/huggingface/speech-to-speech/blob/main/src/speech_to_speech/backend_registry.py#L389-L393) under keys `"responses_api"` and `"chat_completions"` respectively.

---

## Summary

- **vLLM serves the Responses API** at `/v1/responses` when launched with standard OpenAI-compatible flags
- **Match four values** between vLLM and the pipeline: host/port, API key, served model name, and endpoint version
- **CLI flags** for vLLM configuration are centralized in [`responses_api_language_model_arguments.py`](https://github.com/huggingface/speech-to-speech/blob/main/responses_api_language_model_arguments.py)
- **Python API** uses `ResponsesApiLanguageModelHandlerArguments` with `llm_handler="responses_api"`
- **Chat Completions fallback** provides more reliable tool-call streaming via [`chat_completions_language_model.py`](https://github.com/huggingface/speech-to-speech/blob/main/chat_completions_language_model.py)

---

## Frequently Asked Questions

### What vLLM version supports the Responses API endpoint?

vLLM 0.5.0 and later include experimental Responses API support via the `--enable-responses-api` flag or automatic detection when `/v1/responses` is accessed. Verify your build with `curl http://localhost:8000/v1/models` and check that responses appear in the OpenAPI schema.

### Why does my pipeline fail with "model not found" errors?

The `--responses_api_model_name` value must exactly match `--served-model-name` from your vLLM launch command. Mismatches cause 404 errors because vLLM validates model names against its registry before forwarding to the underlying weights.

### Can I use vLLM with audio-native multimodal models in this pipeline?

Yes—set `--responses_api_audio_content_type input_audio` (as shown in the CLI example) when serving audio-native LLMs like GPT-4o-audio or compatible open counterparts. The handler passes audio bytes directly through the Responses API's native audio input fields.

### How do I debug connection issues between the pipeline and vLLM?

Test the vLLM endpoint independently with: `curl http://localhost:8000/v1/responses -H "Authorization: Bearer dummy" -d '{"model":"qwen","input":"test"}'`. If this succeeds but the pipeline fails, verify `--responses_api_api_key` matches the vLLM `--api-key` exactly, including case sensitivity.