How to Integrate the Speech-to-Speech Pipeline with Hugging Face Inference Providers

Set --llm_backend responses-api and configure base_url plus api_key to route the language model step through any Hugging Face Inference endpoint.

The speech-to-speech pipeline from Hugging Face is a modular real-time system connecting voice activity detection, speech-to-text, language model inference, and text-to-speech synthesis. Integrating it with Hugging Face Inference Providers requires selecting the correct backend and configuring endpoint credentials—no changes to other pipeline stages needed.

---

Architecture Overview

The pipeline routes text prompts through a pluggable language model layer. When you select the responses-api backend, the system swaps the local LM handler for ResponsesApiModelHandler, which communicates with any OpenAI-compatible HTTP endpoint—including Hugging Face's Inference API.

Three components cooperate to enable this integration:

• ResponsesApiLanguageModelHandlerArguments — defines CLI flags for endpoint configuration
• get_llm_handler() — instantiates the correct handler based on backend selection
• ResponsesApiModelHandler — executes HTTP requests to the remote inference endpoint

All queue wiring remains identical regardless of backend choice, ensuring clean separation between transport and inference logic.

---

Step 1: Configure the Responses API Backend

The argument dataclass in src/speech_to_speech/arguments_classes/responses_api_language_model_arguments.py exposes these fields:

Field	Purpose	Default
base_url	Full URL of the HF Inference endpoint	None (required)
api_key	Hugging Face API token	None
model_name	Model identifier for the payload	""
chat_size	Maximum conversation turns	16
init_chat_prompt	System prompt for the session	""

Example CLI invocation targeting a public endpoint:

speech-to-speech \
  --llm_backend responses-api \
  --responses_api_base_url https://api-inference.huggingface.co/models/Qwen/Qwen3-4B-Instruct-2507 \
  --responses_api_api_key $HF_API_TOKEN \
  --responses_api_model_name Qwen/Qwen3-4B-Instruct-2507 \
  --responses_api_chat_size 8

---

Step 2: Backend Selection in the Pipeline

In src/speech_to_speech/s2s_pipeline.py, the get_llm_handler() function branches on module_kwargs.llm_backend:

# src/speech_to_speech/s2s_pipeline.py (conceptual)

def get_llm_handler(module_kwargs, responses_api_kwargs, ...):
    if module_kwargs.llm_backend == "responses-api":
        return ResponsesApiModelHandler(**vars(responses_api_kwargs))
    elif module_kwargs.llm_backend == "chat-completions-api":
        return ChatCompletionsApiModelHandler(...)
    else:
        return LanguageModelHandler(...)  # local transformers

The vars() call unpacks all ResponsesApiLanguageModelHandlerArguments fields as keyword arguments to the handler constructor.

---

Step 3: Handler Implementation

ResponsesApiModelHandler resides in src/speech_to_speech/LLM/responses_api_language_model.py and inherits from BaseOpenAICompatibleHandler. Its core responsibilities:

1. Read from text_prompt_queue (STT output)
2. Build OpenAI-compatible chat-completion requests
3. POST to the configured base_url with api_key header
4. Write response text to lm_response_queue (TTS input)

The handler respects the chat_size window, maintaining rolling conversation history across turns.

---

Complete Integration Examples

CLI: Realtime Mode with HF Inference

speech-to-speech \
  --mode realtime \
  --llm_backend responses-api \
  --responses_api_base_url https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.2 \
  --responses_api_api_key $HF_API_TOKEN \
  --responses_api_model_name mistralai/Mistral-7B-Instruct-v0.2 \
  --stt whisper \
  --tts kokoro \
  --vad_handler_kwargs.smart_turn true

Python: Programmatic Configuration

from speech_to_speech.s2s_pipeline import (
    parse_arguments,
    prepare_all_args,
    initialize_queues_and_events,
    build_pipeline,
)

# Parse base configuration

args = parse_arguments()

# Force HF Inference backend

args.module_kwargs.llm_backend = "responses-api"
args.responses_api_language_model_handler_kwargs.base_url = (
    "https://api-inference.huggingface.co/models/Qwen/Qwen3-4B-Instruct-2507"
)
args.responses_api_language_model_handler_kwargs.api_key = "hf_XXXXXXXXXXXXXXXXXXXX"
args.responses_api_language_model_handler_kwargs.model_name = "Qwen/Qwen3-4B-Instruct-2507"

# Initialize pipeline infrastructure

prepare_all_args(
    args.module_kwargs,
    args.whisper_stt_handler_kwargs,
    args.responses_api_language_model_handler_kwargs,
    # ... other handler kwargs

)
queues_and_events = initialize_queues_and_events()

# Build and start

manager = build_pipeline(
    args.module_kwargs,
    args.socket_receiver_kwargs,
    args.socket_sender_kwargs,
    # ... remaining kwargs

    queues_and_events,
)
manager.start()
manager.wait()

Local Audio I/O with Remote LM

speech-to-speech \
  --mode local \
  --llm_backend responses-api \
  --responses_api_base_url https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3-8B-Instruct \
  --responses_api_api_key $HF_API_TOKEN

The --mode local flag keeps audio processing on-device while delegating all language model inference to the remote Hugging Face endpoint.

---

Queue Flow and Realtime Configuration

The pipeline queues are wired in src/speech_to_speech/s2s_pipeline.py within _build_pipeline_handlers():

• text_prompt_queue → LM handler input (STT output)
• lm_response_queue → LM handler output (TTS input)

When running in realtime mode, RealtimeConfig from src/speech_to_speech/api/openai_realtime/runtime_config.py propagates chat parameters (chat_size, init_chat_prompt) to the handler through setup_kwargs, ensuring consistent behavior across backend types.

---

Key Source Files

File Path	Purpose
src/speech_to_speech/s2s_pipeline.py	Central orchestration; backend selection via get_llm_handler()
src/speech_to_speech/LLM/responses_api_language_model.py	ResponsesApiModelHandler implementation
src/speech_to_speech/arguments_classes/responses_api_language_model_arguments.py	CLI argument definitions for HF Inference
src/speech_to_speech/api/openai_realtime/runtime_config.py	Realtime chat configuration
src/speech_to_speech/pipeline/handler_types.py	Type aliases for queue message types

---

Summary

• Select backend: Use --llm_backend responses-api to enable Hugging Face Inference Provider integration
• Configure endpoint: Set base_url and api_key via dedicated CLI flags or programmatic kwargs
• Preserve pipeline: VAD, STT, and TTS components require no changes when switching LM backends
• Leverage compatibility: The responses-api backend uses standard OpenAI chat-completion protocol, supporting any compatible endpoint

---

Frequently Asked Questions

What model formats work with the responses-api backend?

Any model exposed through Hugging Face Inference API or compatible OpenAI chat-completion endpoints works. This includes instruction-tuned LLMs like Mistral, Llama, Qwen, and proprietary models served via Hugging Face's dedicated inference infrastructure.

Do I need GPU resources locally when using HF Inference Providers?

No. The responses-api backend offloads all language model computation to the remote endpoint. Local resources are only consumed by VAD, STT, and TTS stages—which can themselves be configured to use remote services if desired.

How does conversation history persist across turns?

The ResponsesApiModelHandler maintains a rolling window of the last chat_size messages, passed as the messages array in each API request. For stateless endpoints, history is reconstructed client-side; for stateful WebSocket endpoints, additional connection management may apply.

Can I use this backend with third-party OpenAI-compatible APIs?

Yes. The responses-api backend is protocol-compatible with any service implementing OpenAI's chat completions schema. Replace base_url with your provider's endpoint (e.g., https://api.openai.com/v1, https://api.anthropic.com/v1, or self-hosted vLLM instances).

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}