# speech-to-speech | Hugging Face | Knowledge Base | Instagit

Build local voice agents with open-source models

GitHub Stars: 5.6k

Repository: https://github.com/huggingface/speech-to-speech

---

## Articles

### [Speech-to-Speech CLI Commands: Complete Guide to the Hugging Face Toolkit](/huggingface/speech-to-speech/available-cli-commands-speech-to-speech)

Master Hugging Face speech-to-speech CLI commands. Explore run, demo, benchmark, and realtime options for seamless voice interaction. Get started now.

- Tags: how-to-guide
- Published: 2026-08-11

### [How to Run the Hugging Face Speech-to-Speech Pipeline in Offline Mode](/huggingface/speech-to-speech/how-to-run-speech-to-speech-pipeline-offline-mode)

Run the Hugging Face Speech-to-Speech pipeline offline. Cache models locally or use explicit paths to operate without a network connection and maintain your workflow.

- Tags: how-to-guide
- Published: 2026-08-11

### [Supported TTS Backends in Hugging Face Speech-to-Speech and Installation Guide](/huggingface/speech-to-speech/supported-tts-backends-installation)

Discover supported TTS backends in Hugging Face Speech-to-Speech: chatTTS, MMS, pocket, kokoro, and qwen3. Get installation guides for seamless setup.

- Tags: installation-guide
- Published: 2026-08-11

### [How to Select Different TTS Backends for Speech-to-Speech: A Complete Configuration Guide](/huggingface/speech-to-speech/how-to-select-different-tts-backends)

Configure your Speech-to-Speech project easily. Learn how to select TTS backends like MLX GGML or Torch within the huggingface repository for optimal performance.

- Tags: how-to-guide
- Published: 2026-08-11

### [How to Skip STT and Send Audio Directly to an Audio-Capable LLM in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/skip-stt-send-audio-directly-audio-capable-llm)

Learn how to skip STT and send audio directly to audio-capable LLMs in Hugging Face Speech-to-Speech by setting --stt none. Stream raw audio for efficient processing.

- Tags: how-to-guide
- Published: 2026-08-11

### [How to Use the Chat Completions API for LLM Backend in Speech-to-Speech](/huggingface/speech-to-speech/use-chat-completions-api-llm-backend)

Learn how to use the Chat Completions API for your LLM backend with huggingface speech-to-speech. Manage your LLM lifecycle easily and efficiently.

- Tags: how-to-guide
- Published: 2026-08-11

### [How to Use vLLM with the Responses API Backend for Speech-to-Speech Pipelines](/huggingface/speech-to-speech/use-vllm-with-responses-api-backend)

Learn how to use vLLM with the Responses API backend for speech-to-speech pipelines. Point the speech-to-speech library to your vLLM HTTP endpoint using responses_api_base_url and responses_api_model_name.

- Tags: how-to-guide
- Published: 2026-08-11

### [How to Configure the Responses API for LLM Backend in huggingface/speech-to-speech](/huggingface/speech-to-speech/configure-responses-api-llm-backend)

Easily configure the Responses API for your LLM backend in huggingface/speech-to-speech. Learn to set API keys, model names, and endpoint URLs for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-11

### [How to Use an OpenAI-Compatible API for the LLM Backend in speech-to-speech](/huggingface/speech-to-speech/how-to-use-openai-compatible-api-llm-backend)

Integrate an OpenAI-compatible API with the huggingface speech-to-speech pipeline. Connect to Ollama, Azure OpenAI, or self-hosted LLMs using the responses_api backend.

- Tags: how-to-guide
- Published: 2026-08-11

### [How to Select Different LLM Backends for Speech-to-Speech: A Complete Guide](/huggingface/speech-to-speech/how-to-select-different-llm-backends)

Master selecting LLM backends for speech-to-speech with this guide. Easily switch between transformers, mlx-lm, and more using CLI flags or Python functions.

- Tags: how-to-guide
- Published: 2026-08-11

### [How to Use Faster Whisper as an STT Backend in Speech-to-Speech](/huggingface/speech-to-speech/how-to-use-faster-whisper-as-stt-backend)

Integrate Faster Whisper as your STT backend for enhanced speech-to-speech performance. Learn how to set up this powerful tool easily for your projects. Improve your audio processing today.

- Tags: how-to-guide
- Published: 2026-08-11

### [How to Use Whisper as an STT Backend in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-use-whisper-as-stt-backend)

Learn how to use Whisper as an STT backend for your speech-to-speech pipeline. Configure model parameters and generation settings easily via the CLI for efficient transcription.

- Tags: how-to-guide
- Published: 2026-08-11

### [Supported STT Backends in speech-to-speech: Installation and Usage Guide](/huggingface/speech-to-speech/supported-stt-backends-installation)

Explore supported STT backends in huggingface speech-to-speech: Whisper, Faster-Whisper, Parakeet, and more. Learn how to easily install and use them for your projects.

- Tags: how-to-guide
- Published: 2026-08-11

### [How to Select Different STT Backends for Speech-to-Speech: A Complete Guide](/huggingface/speech-to-speech/how-to-select-different-stt-backends)

Effortlessly select STT backends for speech-to-speech using the --stt flag or programmatically. Explore options like whisper, faster-whisper, and mlx-audio-whisper for your Hugging Face pipeline.

- Tags: how-to-guide
- Published: 2026-08-11

### [What Is Smart Turn Endpointing and How to Enable It in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/what-is-smart-turn-endpointing-how-to-enable)

Learn what Smart Turn endpointing is and how to enable it in Hugging Face Speech-to-Speech. Prevent premature responses and cut-offs for better voice assistant interactions.

- Tags: how-to-guide
- Published: 2026-08-11

### [How to Configure VAD Parameters: Threshold, Minimum Speech Duration, and Smart Turn in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/configure-vad-parameters-threshold-min-speech-duration)

Master VAD configuration in Hugging Face Speech-to-Speech. Learn to tune threshold, min speech duration, and smart turn for optimal performance.

- Tags: how-to-guide
- Published: 2026-08-11

### [How the VAD Component Works in Hugging Face Speech-to-Speech: A Deep Dive into Voice Activity Detection](/huggingface/speech-to-speech/how-vad-component-works-speech-to-speech)

Learn how the VAD component in Hugging Face Speech-to-Speech detects speech and converts audio. Understand voice activity detection for improved processing.

- Tags: deep-dive
- Published: 2026-08-11

### [How to Swap Components in the Speech-to-Speech Pipeline: Complete Backend Customization Guide](/huggingface/speech-to-speech/how-to-swap-components-speech-to-speech-pipeline)

Customize the Hugging Face Speech-to-Speech pipeline by swapping components using command-line flags or JSON config. No code changes needed for backend customization.

- Tags: how-to-guide
- Published: 2026-08-11

### [Hugging Face Speech-to-Speech Pipeline Components: A Complete Technical Breakdown](/huggingface/speech-to-speech/what-are-components-of-speech-to-speech-pipeline)

Explore the 8 core components of the Hugging Face Speech-to-Speech pipeline. Get a complete technical breakdown of VAD, STT, LLM, TTS, and more.

- Tags: deep-dive
- Published: 2026-08-11

### [How to Set Up the Speech-to-Speech Pipeline: Complete Installation and Configuration Guide](/huggingface/speech-to-speech/how-to-set-up-speech-to-speech-pipeline)

Learn how to set up the speech-to-speech pipeline with this complete installation and configuration guide. Get real-time voice translation working in minutes.

- Tags: how-to-guide
- Published: 2026-08-11

### [How to Configure the Speech-to-Speech Server for Production Deployment with Authentication](/huggingface/speech-to-speech/configure-server-production-deployment-authentication)

Secure your Hugging Face Speech-to-Speech server for production. Learn how to configure authentication and usage limits with OAuth and environment variables for a robust deployment.

- Tags: how-to-guide
- Published: 2026-08-10

### [How to Monitor and Debug Latency Issues in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/monitor-debug-latency-issues-speech-pipeline)

Effectively monitor and debug speech-to-speech pipeline latency. Learn to track TTFA and RTF using built-in measurements for faster, more responsive applications.

- Tags: tutorial
- Published: 2026-08-10

### [How to Switch Between STT Backends in Hugging Face Speech-to-Speech: Parakeet, Whisper, Faster-Whisper, and Paraformer](/huggingface/speech-to-speech/switch-different-stt-backends-parakeet-whisper-paraformer)

Easily switch STT backends like Parakeet, Whisper, and Paraformer in Hugging Face Speech-to-Speech. Configure the stt field in ModuleArguments and set your preferred STT model.

- Tags: how-to-guide
- Published: 2026-08-10

### [How to Configure ICE Servers for WebRTC in Production Deployments](/huggingface/speech-to-speech/configure-ice-servers-webrtc-production-deployments)

Learn how to configure ICE servers for WebRTC in production. Set the SPEECH_TO_SPEECH_ICE_SERVERS env variable to integrate STUN/TURN servers seamlessly.

- Tags: how-to-guide
- Published: 2026-08-10

### [Supported Audio Formats and Sample Rates in the Hugging Face Speech-to-Speech Pipeline](/huggingface/speech-to-speech/supported-audio-formats-sample-rates-pipeline)

Discover the audio formats and sample rates supported by the Hugging Face Speech-to-Speech pipeline. Learn about native processing and automatic resampling for seamless integration.

- Tags: api-reference
- Published: 2026-08-10

### [How to Deploy Speech-to-Speech Server with Docker and Docker Compose](/huggingface/speech-to-speech/deploy-speech-to-speech-server-docker-docker-compose)

Easily deploy a speech-to-speech server with Docker and Docker Compose. This guide shows how to set up a GPU-accelerated LLM backend for seamless voice translation.

- Tags: how-to-guide
- Published: 2026-08-10

### [How to Integrate vLLM or llama.cpp Servers as an LLM Backend in Speech-to-Speech](/huggingface/speech-to-speech/integrate-vllm-llamacpp-servers-llm-backend)

Learn to integrate vLLM or llama.cpp as your LLM backend in Speech-to-Speech. Set the llm_backend to responses-api or chat-completions and use your server's OpenAI compatible endpoint.

- Tags: how-to-guide
- Published: 2026-08-10

### [How Voice Cloning and Custom Voices Work with Pocket TTS: Implementation Guide](/huggingface/speech-to-speech/voice-cloning-custom-voices-pocket-tts)

Learn how Pocket TTS enables voice cloning and custom voices using preset embeddings, local files, or Hugging Face references. Implement your own unique speech generation today.

- Tags: how-to-guide
- Published: 2026-08-10

### [How to Configure Multi-Language Support with Automatic Language Detection in Speech-to-Speech](/huggingface/speech-to-speech/configure-multi-language-support-automatic-language-detection)

Configure multi-language speech-to-speech translation with automatic language detection for 25+ European languages. Install lingua-py and set language to None for STT handlers.

- Tags: how-to-guide
- Published: 2026-08-10

### [How to Configure CUDA Versions for Qwen3-TTS on Linux Systems](/huggingface/speech-to-speech/configure-cuda-versions-qwen3-tts-linux)

Configure CUDA for Qwen3-TTS on Linux. Install CUDA 11.8+ or 12.0+, then accelerate Qwen3-TTS with `faster-qwen3-tts[ggml]` on your GPU.

- Tags: how-to-guide
- Published: 2026-08-10

### [How to Configure Smart Turn Endpointing for Better Turn-Taking](/huggingface/speech-to-speech/configure-smart-turn-endpointing-better-turn-taking)

Learn to configure Smart Turn endpointing with ONNX models to improve turn-taking in your speech-to-speech applications and prevent interruptions. Optimize conversational flow.

- Tags: how-to-guide
- Published: 2026-08-10

### [How OpenAI Realtime WebSocket Protocol Works with the Speech-to-Speech Server](/huggingface/speech-to-speech/openai-realtime-websocket-protocol-speech-to-speech-server)

Understand how OpenAI Realtime WebSocket protocol powers speech-to-speech conversations. Learn about its STT LM TTS processing chains and efficient audio streaming for real-time interaction.

- Tags: how-to-guide
- Published: 2026-08-10

### [How to Implement a Custom STT Backend for the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/implement-custom-stt-backend-speech-to-speech-pipeline)

Learn to implement a custom STT backend for the speech-to-speech pipeline. This guide details creating configuration dataclasses, handler classes, and registering your backend for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-10

### [Handling Model Caching and HF_HUB_OFFLINE for Air-Gapped Deployments in Speech-to-Speech](/huggingface/speech-to-speech/handling-model-caching-hf-hub-offline-air-gapped)

Deploy speech-to-speech models offline in air-gapped environments. Learn to handle model caching and HF_HUB_OFFLINE for seamless operation without network access.

- Tags: how-to-guide
- Published: 2026-08-09

### [Optimizing Apple Silicon Performance with mlx-lm and mlx-audio Backends](/huggingface/speech-to-speech/optimizing-apple-silicon-performance-mlx)

Boost Apple Silicon performance using mlx-lm and mlx-audio backends for speech-to-speech. Optimize inference speed and accelerate your ML workflows.

- Tags: performance
- Published: 2026-08-09

### [Setting Up HF Inference Providers with OpenAI-Compatible Endpoints in Speech-to-Speech](/huggingface/speech-to-speech/setting-up-hf-inference-providers-openai-compatible)

Easily set up HF Inference Providers with OpenAI-compatible endpoints for speech-to-speech. Configure LLM backend, API base URL, and API key for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-09

### [Implementing Tool Calls and Function Calling in Voice Conversations with Speech-to-Speech](/huggingface/speech-to-speech/implementing-tool-calls-function-calling-voice)

Learn how to implement tool calls and function calling in voice conversations using the Speech-to-Speech pipeline. Stream function invocations for structured tool execution.

- Tags: how-to-guide
- Published: 2026-08-09

### [Configuring Reasoning Effort and Thinking Mode Suppression for Voice Responses in Speech-to-Speech](/huggingface/speech-to-speech/configuring-reasoning-effort-thinking-mode-suppression)

Control reasoning effort and thinking mode for voice responses in Hugging Face Speech-to-Speech. Learn to set backend arguments and process output effectively.

- Tags: how-to-guide
- Published: 2026-08-09

### [Implementing Audio Input Directly to LLM Without STT: The Hugging Face Speech-to-Speech Architecture](/huggingface/speech-to-speech/implementing-audio-input-directly-to-llm-no-stt)

Bypass STT in Hugging Face speech-to-speech. Route raw audio directly to LLM with --stt none and an audio-capable backend for efficient audio input.

- Tags: architecture
- Published: 2026-08-09

### [Migrating from the Deprecated `--mode` Flag to `serve`, `talk`, and `local` Commands in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/migrating-deprecated-mode-to-new-commands)

Migrate from the deprecated --mode flag to Hugging Face Speech-to-Speech serve talk and local commands. Learn how to update your CLI usage for efficient speech processing.

- Tags: migration-guide
- Published: 2026-08-09

### [Setting Up Docker Deployment with GPU Passthrough for Production](/huggingface/speech-to-speech/setting-up-docker-deployment-gpu-passthrough)

Deploy Hugging Face Speech-to-Speech with GPU passthrough in production. Configure docker-compose.yml to enable CUDA access for LLM and pipeline containers.

- Tags: how-to-guide
- Published: 2026-08-09

### [Implementing the Backend Registry Pattern for Swappable Pipeline Components](/huggingface/speech-to-speech/implementing-backend-registry-pattern)

Learn how the huggingface/speech-to-speech library uses a backend registry pattern for swappable STT, LLM, and TTS components at runtime. Explore its implementation and benefits.

- Tags: architecture
- Published: 2026-08-09

### [Debugging Live Transcription Streaming and TranscriptionNotifier Events in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/debugging-live-transcription-streaming-notifier)

Debug live transcription streaming and TranscriptionNotifier events in Hugging Face Speech-to-Speech. Understand Partial and Completed events for real-time STT updates.

- Tags: debugging
- Published: 2026-08-09

### [Implementing Custom Voice Cloning with Pocket TTS in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/implementing-custom-voice-cloning-pocket-tts)

Implement custom voice cloning with Pocket TTS in the speech-to-speech pipeline. Generate speaker embeddings and use VoicePrompt with PocketTTSHandler to synthesize audio in your target voice.

- Tags: how-to-guide
- Published: 2026-08-09

### [Managing Pipeline Thread Pools and Queue Sizes for Concurrent Sessions in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/managing-pipeline-thread-pools-queue-sizes)

Optimize concurrent speech-to-speech sessions by managing pipeline thread pools and queue sizes. Learn how to configure --num_pipelines for efficient handler threads in Hugging Face speech-to-speech.

- Tags: performance
- Published: 2026-08-09

### [Setting Up WebRTC Sessions for Browser-Based Voice Clients](/huggingface/speech-to-speech/setting-up-webrtc-sessions-browser-clients)

Learn how to set up WebRTC sessions for browser-based voice clients using the Hugging Face speech-to-speech library. Enable seamless peer-to-peer voice streaming with aiortc.

- Tags: how-to-guide
- Published: 2026-08-09

### [Configuring Qwen3-TTS Backend with GGML vs MLX Quantization: A Complete Guide](/huggingface/speech-to-speech/configuring-qwen3-tts-ggml-mlx-quantization)

Compare GGML vs MLX quantization for Qwen3-TTS backend. Learn how to configure and optimize your speech-to-speech model on Apple Silicon, Linux, and Windows.

- Tags: deep-dive
- Published: 2026-08-09

### [Implementing Voice Activity Detection Interrupt for Turn-Taking in Speech-to-Speech](/huggingface/speech-to-speech/implementing-vad-interrupt-for-turn-taking)

Implement voice activity detection interrupt for real-time turn-taking in speech-to-speech with Hugging Face's VADHandler. Control responsiveness and prevent false triggers.

- Tags: how-to-guide
- Published: 2026-08-09

### [Supporting Multi-Language Voice Conversations with Automatic Language Detection in Speech-to-Speech](/huggingface/speech-to-speech/supporting-multi-language-voice-conversations)

Enable multi-language voice conversations with automatic language detection in speech-to-speech. Our pipeline supports ISO-639-1 codes for seamless LLM and TTS coordination.

- Tags: deep-dive
- Published: 2026-08-09

### [Troubleshooting VAD Threshold and min_speech_ms for Noisy Environments in Speech-to-Speech](/huggingface/speech-to-speech/troubleshooting-vad-threshold-min-speech-ms)

Troubleshoot VAD threshold and min_speech_ms for noisy environments in Hugging Face Speech-to-Speech. Boost accuracy by tuning parameters for better speech detection.

- Tags: troubleshooting
- Published: 2026-08-09

### [How to Run Speech-to-Speech Fully Offline with a Local llama.cpp Server](/huggingface/speech-to-speech/running-speech-to-speech-offline-llama-cpp)

Learn to run speech-to-speech fully offline with a local llama.cpp server. Control Hugging Face’s pipeline locally using simple flags for enhanced privacy and speed.

- Tags: how-to-guide
- Published: 2026-08-09

### [Optimizing Latency with Speculative Turn Processing and Barge-In Handling in Speech-to-Speech](/huggingface/speech-to-speech/optimizing-latency-speculative-turn-barge-in)

Optimize speech-to-speech latency with speculative turn processing and barge-in handling. Achieve sub-second responses in real-time dialogue. Learn how at huggingface/speech-to-speech.

- Tags: performance
- Published: 2026-08-09

### [Implementing WebSocket Client Connections with OpenAI Realtime Protocol in the Speech-to-Speech Library](/huggingface/speech-to-speech/implementing-websocket-client-openai-realtime)

Learn to implement WebSocket client connections with the OpenAI Realtime protocol in the huggingface/speech-to-speech library. Stream audio and manage sessions effectively.

- Tags: how-to-guide
- Published: 2026-08-09

### [Setting Up LLM Proxy for Concurrent Background Tasks in Voice Conversations: A Complete Guide](/huggingface/speech-to-speech/setting-up-llm-proxy-for-concurrent-tasks)

Set up an LLM proxy for concurrent background tasks in voice conversations. This guide explains how to use the OpenAI compatible passthrough for parallel audio processing and external LLM requests.

- Tags: how-to-guide
- Published: 2026-08-09

### [Configuring Smart Turn for End-of-Speech Detection Accuracy in Speech-to-Speech Systems](/huggingface/speech-to-speech/configuring-smart-turn-for-speech-detection-accuracy)

Improve speech-to-speech accuracy using Smart Turn for end-of-speech detection. Learn how this ONNX classifier enhances turn completion with acoustic and linguistic analysis.

- Tags: tutorial
- Published: 2026-08-09

### [How to Implement Custom STT or TTS Backends in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-implement-custom-stt-tts-backends-in-pipeline)

Learn to implement custom STT or TTS backends in the Hugging Face speech-to-speech pipeline. Discover how to create handler classes, define configurations, and register your custom backend for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-09

### [How the Speech-to-Speech Server Routes WebSocket Connections to Pipeline Instances in the Pool](/huggingface/speech-to-speech/server-route-websocket-connections-pipeline-instances-pool)

Learn how the speech-to-speech server routes WebSocket connections to available pipeline instances. Discover the efficient unit claiming and session binding process.

- Tags: internals
- Published: 2026-08-08

### [How the Speech-to-Speech Pipeline Queue Architecture Prevents Head-of-Line Blocking](/huggingface/speech-to-speech/pipeline-queue-architecture-prevent-head-of-line-blocking)

Learn how the huggingface speech-to-speech pipeline queue architecture prevents head-of-line blocking. Discover isolated data flow and speculative turn processing for efficient audio conversion.

- Tags: internals
- Published: 2026-08-08

### [How to Implement Custom Transcription Notifiers for Downstream Processing in Speech-to-Speech](/huggingface/speech-to-speech/implement-custom-transcription-notifiers-downstream-processing)

Learn to implement custom transcription notifiers for downstream processing in Hugging Face speech-to-speech. Subclass BaseHandler and override process for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-08

### [Required NLTK Resources for Hugging Face Speech-to-Speech: Offline Caching Guide](/huggingface/speech-to-speech/required-nltk-resources-cached-offline-use)

Discover the two essential NLTK resources for Hugging Face Speech-to-Speech. Learn how to automatically cache them for seamless offline execution.

- Tags: how-to-guide
- Published: 2026-08-08

### [How the `--stt none` Mode Works with Audio-Input Capable LLM Models](/huggingface/speech-to-speech/stt-none-mode-audio-input-llm-models)

Understand how the --stt none mode bypasses speech-to-text, sending raw audio to LLMs that support native audio input for direct processing.

- Tags: internals
- Published: 2026-08-08

### [Difference Between `--min_speech_ms` and `--min_speech_continuation_ms` VAD Parameters](/huggingface/speech-to-speech/differences-min_speech_ms-min_speech_continuation_ms-vad-parameters)

Understand the difference between min_speech_ms and min_speech_continuation_ms VAD parameters. Learn how these settings control speech turn detection and prevent utterance splitting in speech-to-speech.

- Tags: deep-dive
- Published: 2026-08-08

### [How to Use Custom Voice Files with Pocket TTS for Voice Cloning](/huggingface/speech-to-speech/use-custom-voice-files-pocket-tts-voice-cloning)

Learn to voice clone with custom audio files using Pocket TTS. Easily integrate your own voice data into the Hugging Face speech-to-speech model. Get started now.

- Tags: how-to-guide
- Published: 2026-08-08

### [Cancel Scope Mechanism in Hugging Face Speech-to-Speech: Handling Partial Responses During Interruption](/huggingface/speech-to-speech/cancel-scope-mechanism-handle-partial-responses-interruption)

Discover the cancel scope mechanism in Hugging Face Speech-to-Speech. Learn how it instantly halts stale pipeline operations and filters partial responses during user interruptions.

- Tags: internals
- Published: 2026-08-08

### [How to Configure Live Transcription for Multi-Pipeline Deployments on macOS](/huggingface/speech-to-speech/configure-live-transcription-multi-pipeline-macos)

Learn how to configure live transcription for multi-pipeline deployments on macOS. Avoid MLX global lock issues by managing single pipeline instances or disabling live transcription.

- Tags: how-to-guide
- Published: 2026-08-08

### [Memory Consumption Characteristics of STT Backends in speech-to-speech: Parakeet, Whisper, and Faster Whisper](/huggingface/speech-to-speech/memory-consumption-stt-backends-parakeet-whisper-faster-whisper)

Compare memory consumption of Parakeet, Whisper, and Faster Whisper STT backends. Discover RAM and GPU usage for speech-to-speech models.

- Tags: performance
- Published: 2026-08-08

### [How mlx_lock Serialization Works on Apple Silicon and When to Disable It](/huggingface/speech-to-speech/mlx_lock-serialization-apple-silicon-disable-when)

Understand mlx_lock serialization on Apple Silicon, how it prevents race conditions in MLX GPU operations, and when disabling it boosts performance.

- Tags: internals
- Published: 2026-08-08

### [How to Add a New TTS Backend to Speech-to-Speech: Handler Pattern Guide](/huggingface/speech-to-speech/add-new-tts-backend-handler-pattern)

Easily add a new TTS backend to Speech-to-Speech. Learn the handler pattern for CLI configuration, handler implementation, and backend registration with our step-by-step guide.

- Tags: how-to-guide
- Published: 2026-08-08

### [OpenAI Realtime-Compatible Server WebSocket Event Types: Complete Reference](/huggingface/speech-to-speech/websocket-event-types-openai-realtime-server)

Explore all 20 OpenAI Realtime WebSocket event types implemented by Hugging Face speech-to-speech. Get a complete reference for streaming audio, managing conversations, and LLM responses.

- Tags: api-reference
- Published: 2026-08-08

### [How the Speech-to-Speech Pipeline Handles Audio Format Conversion Between Different Component Sample Rates](/huggingface/speech-to-speech/pipeline-audio-format-conversion-component-sample-rates)

Learn how the Hugging Face Speech-to-Speech pipeline resamples audio between 16 kHz and other sample rates for seamless component integration. Discover the methods used for efficient conversion.

- Tags: internals
- Published: 2026-08-08

### [How to Configure DeepFilterNet for VAD Audio Enhancement Without NumPy Conflicts](/huggingface/speech-to-speech/configure-deepfilternet-vad-enhancement-pocket-tts-numpy-conflict)

Configure DeepFilterNet for VAD audio enhancement. Avoid NumPy conflicts by installing the df package and managing NumPy versions. Enable audio enhancement with a simple flag.

- Tags: how-to-guide
- Published: 2026-08-08

### [GGML vs mlx-audio for Qwen3-TTS on Apple Silicon: Performance Analysis](/huggingface/speech-to-speech/performance-ggml-vs-mlx-audio-qwen3-tts-apple-silicon)

Analyze GGML vs mlx-audio for Qwen3-TTS on Apple Silicon. Discover real-time streaming with mlx-audio's Metal GPU acceleration compared to GGML's CPU latency.

- Tags: performance
- Published: 2026-08-08

### [How the Speculative Turn Tracker Handles Turn Revisions During User Interruptions](/huggingface/speech-to-speech/speculative-turn-tracker-handle-turn-revisions-user-interrupts)

Learn how the speculative turn tracker ensures audio consistency by managing turn revisions during user interruptions with monotonically increasing revision numbers.

- Tags: internals
- Published: 2026-08-08

### [How to Integrate a Custom STT Backend into the Speech-to-Speech Backend Registry](/huggingface/speech-to-speech/integrate-custom-stt-backend-registry-system)

Integrate a custom STT backend into the Speech-to-Speech backend registry. Learn how to create a BackendSpec entry to link your dataclass and handler factory.

- Tags: how-to-guide
- Published: 2026-08-08

### [Understanding the Difference Between responses-api and chat-completions LLM Backends for Tool-Call Streaming](/huggingface/speech-to-speech/differences-responses-api-chat-completions-llm-backends-tool-call-streaming)

Understand responses-api vs chat-completions LLM backends for tool call streaming. Learn how each streams tool calls differently for efficient integration.

- Tags: deep-dive
- Published: 2026-08-08

### [How the LLM Proxy Handles Authentication and Rate Limiting for Side Tasks in Speech-to-Speech](/huggingface/speech-to-speech/llm-proxy-authentication-rate-limiting-side-tasks)

Discover how the LLM proxy in speech-to-speech handles authentication and rate limiting by delegating to external gateways and using upstream API keys for secure, controlled access.

- Tags: internals
- Published: 2026-08-08

### [Thread Safety Considerations for Multiple Pipeline Instances with `--num_pipelines`](/huggingface/speech-to-speech/thread-safety-multiple-pipeline-instances-num_pipelines)

Learn about thread safety for multiple speech-to-speech pipeline instances with --num_pipelines. Understand isolated state and Apple Silicon MLX lock implications for efficient inference.

- Tags: internals
- Published: 2026-08-08

### [How to Configure Smart Turn Endpointing Parameters to Reduce False Speech Finalizations in Noisy Environments](/huggingface/speech-to-speech/configure-smart-turn-endpointing-reduce-false-finalizations-noisy-environments)

Reduce false speech finalizations in noisy environments by configuring Smart Turn endpointing parameters. Adjust thresholds and delays for better accuracy with our expert guide.

- Tags: how-to-guide
- Published: 2026-08-08

### [How the VAD → STT → LLM → TTS Pipeline Handles Concurrent Audio Streams in Multi-Client Scenarios](/huggingface/speech-to-speech/vad-stt-llm-tts-pipeline-concurrent-audio-streams-multi-client)

Learn how the VAD STT LLM TTS pipeline manages concurrent audio streams for multiple clients. Discover session isolation, concurrency limits, and safe thread reuse for efficient handling.

- Tags: architecture
- Published: 2026-08-08

### [How to Benchmark TTS Quantization Variants on macOS: Complete Guide](/huggingface/speech-to-speech/how-to-benchmark-tts-quantization-variants-on-macos)

Benchmark TTS quantization variants on macOS with Qwen-3-TTS. Compare bf16, 4bit, 6bit, and 8bit levels by latency, RTF, and warm-up time. Run the benchmark script now.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Handle MLX Lock Contention on Apple Silicon with Multiple Pipelines](/huggingface/speech-to-speech/how-to-handle-mlx-lock-contention-on-apple-silicon-with-multiple-pipelines)

Learn how to manage MLX lock contention on Apple Silicon for multiple pipelines. Optimize your MLX GPU operations and avoid Metal command buffer corruption with our expert guide.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Use Vision Language Models with the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-use-vision-language-models-with-the-speech-to-speech-pipeline)

Integrate vision language models into the Speech-to-Speech pipeline with ease. Learn how to set llm_is_vlm=True and provide image inputs for enhanced speech generation. Explore the huggingface speech-to-speech repository.

- Tags: how-to-guide
- Published: 2026-08-07

### [`responses-api` vs `chat-completions` in Hugging Face Speech-to-Speech: Complete Backend Guide](/huggingface/speech-to-speech/what-are-the-differences-between-llm_backend-responses-api-and-chat-completions)

Understand responses-api vs chat-completions in Hugging Face Speech-to-Speech. Use responses-api for OpenAI Realtime and chat-completions for modern tool-call streaming with vLLM compatibility.

- Tags: deep-dive
- Published: 2026-08-07

### [How to Migrate from Deprecated `--mode` Flags to New `serve`/`local` Commands in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-to-migrate-from-deprecated-mode-flags-to-new-serve-local-commands)

Easily migrate from deprecated --mode flags to new serve and local commands in Hugging Face Speech-to-Speech. Update your workflow and keep all arguments the same.

- Tags: migration-guide
- Published: 2026-08-07

### [How to Set Up a Docker Compose Environment with Local LLM Server for Speech-to-Speech](/huggingface/speech-to-speech/how-to-set-up-docker-compose-environment-with-local-llm-server)

Easily set up a Docker Compose environment with a local LLM server for speech-to-speech using the huggingface/speech-to-speech repository. Get GPU support out of the box.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Configure VAD Threshold and Silence Parameters for Different Acoustic Environments in huggingface/speech-to-speech](/huggingface/speech-to-speech/how-to-configure-vad-threshold-and-silence-parameters-for-different-acoustic-environments)

Configure VAD threshold and silence parameters for optimal speech-to-speech performance in noisy environments. Adjust thresh and min_silence_ms using VADHandlerArguments or runtime updates.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Implement Tool Calling and Function Calling with the Speech-to-Speech Voice Pipeline](/huggingface/speech-to-speech/how-to-implement-tool-calling-and-function-calling-with-the-voice-pipeline)

Learn to implement tool calling and function calling in the Hugging Face Speech-to-Speech pipeline. Discover how function calls are parsed and validated for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Debug Thread Management and Queue-Based Communication in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-debug-thread-management-and-queue-based-communication-in-the-pipeline)

Debug thread management and queue-based communication in Hugging Face's speech-to-speech pipeline. Learn to diagnose deadlocks and hanging threads using lifecycle hooks and instrumentation.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Use Direct Audio Input Mode in speech-to-speech to Send Audio Directly to LLM (Skipping STT)](/huggingface/speech-to-speech/how-to-use-direct-audio-input-mode-to-send-audio-directly-to-llm-without-stt)

Learn how to use direct audio input mode in speech-to-speech to send audio directly to LLM skipping STT. Configure RuntimeConfig for seamless audio routing.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Configure Device Placement for STT, LLM, and TTS on Apple Silicon (MPS)](/huggingface/speech-to-speech/how-to-configure-device-placement-for-stt-llm-tts-on-apple-silicon-with-mps)

Configure STT, LLM, and TTS device placement on Apple Silicon MPS. Use mac optimal settings for auto MPS or specify devices like --stt_device for fine-tuned control.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Implement Barge-In to Interrupt Assistant Responses in the Hugging Face Speech-to-Speech Repository](/huggingface/speech-to-speech/how-to-implement-barge-in-to-interrupt-assistant-responses)

Implement barge-in to interrupt assistant responses in Hugging Face Speech-to-Speech. Learn how VAD, runtime flags, and control messages ensure clean turn transitions for seamless interaction.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Handle CUDA Version Mismatches with Qwen3-TTS on Linux](/huggingface/speech-to-speech/how-to-handle-cuda-version-mismatches-with-qwen3-tts-on-linux)

Resolve Qwen3-TTS CUDA version mismatches on Linux. Install the correct qwentts-cpp-python wheel before speech-to-speech for seamless GGML backend loading and prevent errors.

- Tags: how-to-guide
- Published: 2026-08-07

### [Which TTS Backends Support Voice Cloning in huggingface/speech-to-speech and How to Configure Them](/huggingface/speech-to-speech/which-tts-backends-support-voice-cloning-and-how-to-configure-them)

Discover which TTS backends support voice cloning in huggingface/speech-to-speech and learn how to configure them using reference audio or speaker embeddings.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Optimize Performance with Multiple Pipeline Instances Using `--num_pipelines` in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-to-optimize-performance-with-multiple-pipeline-instances-using-num_pipelines)

Optimize Hugging Face Speech-to-Speech performance by running multiple pipeline instances using --num_pipelines. Boost throughput with isolated state and dedicated threads.

- Tags: performance
- Published: 2026-08-07

### [How to Set Up a Fully Local Voice Agent with llama.cpp and Gemma 4](/huggingface/speech-to-speech/how-to-set-up-a-fully-local-voice-agent-with-llamacpp-and-gemma-4)

Set up a fully local voice agent with llama.cpp and Gemma 4. Run an offline voice assistant on your hardware, bypassing cloud APIs entirely for complete privacy and control.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Use the LLM Proxy for Concurrent Background Tasks in Speech-to-Speech](/huggingface/speech-to-speech/how-to-use-the-llm-proxy-feature-for-concurrent-background-tasks)

Learn how to use the LLM proxy for concurrent background tasks in Huggingface Speech-to-Speech. Enable non-blocking execution and streamline your audio pipeline.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Configure Multi-Language Support with Automatic Language Detection in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-to-configure-multi-language-support-with-automatic-language-detection)

Easily configure multi-language support with automatic language detection in Hugging Face Speech-to-Speech. Install lingua-py and set language to None for 25+ languages.

- Tags: how-to-guide
- Published: 2026-08-07

### [What Is the Speculative Turns Feature for Handling Turn Revisions in Speech-to-Speech?](/huggingface/speech-to-speech/what-is-the-speculative-turns-feature-for-handling-turn-revisions)

Explore speculative turns for speech-to-speech pipelines. This feature efficiently manages audio turn revisions in real-time, ensuring only the latest valid audio is processed.

- Tags: deep-dive
- Published: 2026-08-07

### [How to Enable Live Transcription with Partial Transcript Streaming in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-to-enable-live-transcription-with-partial-transcript-streaming)

Learn to enable live transcription with partial transcript streaming in Hugging Face Speech-to-Speech. Adjust update intervals for real-time transcriptions and enhance your STT workflow.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Swap LLM Backends in Speech-to-Speech: OpenAI, vLLM, llama.cpp, and Hugging Face](/huggingface/speech-to-speech/how-to-swap-llm-backends-openai-api-vllm-llamacpp-hugging-face)

Easily swap LLM backends like OpenAI, vLLM, llama.cpp, and Hugging Face for your speech-to-speech models using the --llm_backend CLI flag and configure your endpoint or model path.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Implement OpenAI Realtime WebSocket Protocol for Voice Agents](/huggingface/speech-to-speech/how-to-implement-openai-realtime-websocket-protocol-for-voice-agents)

Implement OpenAI Realtime WebSocket protocol for voice agents using RealtimeService WebSocketTransport and PipelineUnit for seamless VAD STT LLM TTS streaming.

- Tags: how-to-guide
- Published: 2026-08-07

### [How to Configure Smart Turn Endpointing for Voice Activity Detection in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-do-i-configure-smart-turn-endpointing-for-voice-activity-detection)

Learn to configure Smart Turn endpointing for voice activity detection in Hugging Face Speech-to-Speech. Adjust VAD parameters for better speech processing.

- Tags: how-to-guide
- Published: 2026-08-07

### [How the VAD to STT to LLM to TTS Pipeline Works in speech-to-speech](/huggingface/speech-to-speech/how-does-the-vad-to-stt-to-llm-to-tts-pipeline-architecture-work-in-speech-to-speech)

Understand the VAD to STT to LLM to TTS pipeline for real-time speech-to-speech. Learn how this asynchronous architecture processes live audio through four stages for seamless conversation.

- Tags: architecture
- Published: 2026-08-07

### [How to Configure Kokoro, Pocket TTS, or ChatTTS Backends in the Speech-to-Speech Library](/huggingface/speech-to-speech/how-to-configure-kokoro-pocket-tts-or-chattss-backends)

Easily configure Kokoro, Pocket TTS, or ChatTTS backends in the speech-to-speech library. Use the --tts flag and backend-specific arguments to customize your TTS setup. Learn how now.

- Tags: how-to-guide
- Published: 2026-08-06

### [How to Handle Audio Format Conversion Between Pipeline Stages in Speech-to-Speech](/huggingface/speech-to-speech/how-to-handle-audio-format-conversion-between-pipeline-stages)

Learn how the Speech-to-Speech framework seamlessly converts audio formats between pipeline stages. Discover the five core transformations for efficient 16 kHz audio processing.

- Tags: how-to-guide
- Published: 2026-08-06

### [How to Use Voice Prompts to Customize LLM Behavior in the Hugging Face Speech-to-Speech Repository](/huggingface/speech-to-speech/how-to-use-voice-prompts-to-customize-llm-behavior)

Learn how to use voice prompts to customize LLM behavior in Hugging Face's Speech-to-Speech repository. Control spoken output with lead, session prompt, tools, and tail.

- Tags: how-to-guide
- Published: 2026-08-06

### [How to Implement a Custom STT Handler for New Models in HuggingFace Speech‑to‑Speech](/huggingface/speech-to-speech/how-to-implement-a-custom-stt-handler-for-new-models)

Learn to implement a custom STT handler for new speech-to-speech models in Hugging Face. Subclass BaseHandler, load your model in setup(), and process audio for transcriptions in s2s_pipeline.py.

- Tags: how-to-guide
- Published: 2026-08-06

### [How to Configure Generation Parameters in Huggingface Speech-to-Speech: Temperature and Max Tokens](/huggingface/speech-to-speech/how-to-configure-generation-parameters-like-temperature-and-max-tokens)

Master Huggingface Speech-to-Speech generation parameters like temperature and max tokens. Learn to configure llm_gen_temperature and audio_max_tokens for optimal results in your S2SPipeline.

- Tags: how-to-guide
- Published: 2026-08-06

### [How to Use Tool Calls With the OpenAI Realtime Protocol: A Complete Implementation Guide](/huggingface/speech-to-speech/how-to-use-tool-calls-with-the-openai-realtime-protocol)

Master tool calls with the OpenAI Realtime protocol. Learn the three-layer pipeline for function calls using FunctionTool, build system prompts, and parse output for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-06

### [How Speculative Turns Allow Early LLM Processing in Real-Time Speech-to-Speech Systems](/huggingface/speech-to-speech/how-do-speculative-turns-allow-early-llm-processing)

Discover how speculative turns enable early LLM processing in real-time speech-to-speech systems by starting inference on partial transcriptions to reduce latency.

- Tags: deep-dive
- Published: 2026-08-06

### [How to Configure CUDA and Device Settings Across Components in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-to-configure-cuda-and-device-settings-across-components)

Master CUDA and device settings in Hugging Face speech-to-speech. Learn to assign device IDs and configure multi-GPU scheduling for optimal performance.

- Tags: how-to-guide
- Published: 2026-08-06

### [How to Deploy the Hugging Face Speech‑to‑Speech Pipeline with Docker and Docker Compose](/huggingface/speech-to-speech/how-to-deploy-the-pipeline-with-docker-and-docker-compose)

Deploy the Hugging Face speech-to-speech pipeline easily with Docker and Docker Compose. Launch an LLM backend with GPU support and persistent caching using the provided docker-compose.yml.

- Tags: how-to-guide
- Published: 2026-08-06

### [How to Choose Between `responses-api` and `chat-completions` LLM Backends in Speech-to-Speech](/huggingface/speech-to-speech/how-to-choose-between-responses-api-and-chat-completions-llm-backends)

Choose responses-api for simple OpenAI streaming or chat-completions for reliable tool-call streaming with vLLM/Qwen and advanced reasoning control.

- Tags: best-practices
- Published: 2026-08-06

### [How Turn-Taking and Interruption Handling Work in Real-Time Mode for Speech-to-Speech](/huggingface/speech-to-speech/how-does-turn-taking-and-interruption-handling-work-in-realtime-mode)

Understand real-time speech-to-speech turn-taking and interruption handling. Learn how VAD events, active responses, and session configurations enable seamless interaction.

- Tags: internals
- Published: 2026-08-06

### [How to Implement a Custom TTS Backend Handler in the Hugging Face Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-implement-a-custom-tts-backend-handler)

Learn to implement a custom TTS backend handler in Hugging Face's speech-to-speech pipeline. Extend BaseHandler, define setup and process, and register your handler for custom TTS solutions.

- Tags: how-to-guide
- Published: 2026-08-06

### [How to Debug Audio Pipeline Latency and Dropped Frames in Speech-to-Speech: A Complete Guide](/huggingface/speech-to-speech/how-to-debug-audio-pipeline-latency-and-dropped-frames)

Debug audio pipeline latency and dropped frames in speech-to-speech. Monitor queues, log debug info, and inspect TTS handlers to pinpoint root causes like inference, network issues, or blocksize problems.

- Tags: how-to-guide
- Published: 2026-08-06

### [How WebSocket and WebRTC Transports Handle Audio Streaming in the Speech-to-Speech Library](/huggingface/speech-to-speech/how-do-websocket-and-webrtc-transports-handle-audio-streaming)

Discover how WebSocket and WebRTC handle audio streaming in the Speech-to-Speech library. Learn about their distinct transport mechanisms for real-time voice data.

- Tags: internals
- Published: 2026-08-06

### [How to Customize TTS Voice and Speaker Settings in the Hugging Face Speech‑to‑Speech Repository](/huggingface/speech-to-speech/how-to-customize-tts-voice-and-speaker-settings)

Easily customize TTS voice and speaker settings in the Hugging Face speech-to-speech repository. Learn to configure voice, speaker, language, and cloning arguments with our modular handler pattern.

- Tags: how-to-guide
- Published: 2026-08-06

### [How to Set Up Local Inference with llama.cpp or vLLM in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-to-set-up-local-inference-with-llamacpp-or-vllm)

Set up local inference with llama.cpp or vLLM for Hugging Face Speech-to-Speech. Configure the responses_api_base_url for fully local LLM inference.

- Tags: how-to-guide
- Published: 2026-08-06

### [How to Connect OpenAI Realtime Clients to the Speech-to-Speech WebSocket Server](/huggingface/speech-to-speech/how-to-connect-openai-realtime-clients-to-the-websocket-server)

Connect OpenAI Realtime clients to the Speech-to-Speech WebSocket server using the RealtimeClient SDK. Stream audio and register callbacks for seamless voice interaction.

- Tags: how-to-guide
- Published: 2026-08-06

### [How Smart Turn Endpointing Validates End-of-Speech Decisions: A Deep Dive into Hugging Face's Speech-to-Speech VAD](/huggingface/speech-to-speech/how-does-smart-turn-endpointing-validate-end-of-speech-decisions)

Learn how Smart Turn endpointing validates end-of-speech decisions using energy-based VAD and frame counts. Discover the Hugging Face speech-to-speech method.

- Tags: deep-dive
- Published: 2026-08-06

### [How to Swap STT Backends Between Parakeet, Whisper, Faster Whisper, and Paraformer in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-to-swap-stt-backends-between-parakeet-whisper-faster-whisper-and-paraformer)

Easily swap STT backends in Hugging Face speech-to-speech between Parakeet, Whisper, Faster Whisper, and Paraformer by setting the stt field in ModuleArguments. Build your pipeline now.

- Tags: how-to-guide
- Published: 2026-08-06

### [How to Configure VAD Thresholds and Minimum Speech Duration in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-to-configure-vad-thresholds-and-minimum-speech-duration-parameters)

Learn to configure VAD thresholds and minimum speech duration in Hugging Face Speech-to-Speech. Master these settings for precise audio processing and optimize your pipelines for accurate speech detection.

- Tags: how-to-guide
- Published: 2026-08-06

### [Common Issues and How to Troubleshoot Them in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/common-issues-and-how-to-troubleshoot-them-in-speech-to-speech-pipeline)

Troubleshoot common speech-to-speech pipeline issues like concurrency bugs, dependency mismatches, and protocol errors. Learn diagnostic patterns and targeted logging for the huggingface/speech-to-speech repository.

- Tags: how-to-guide
- Published: 2026-08-05

### [Speech-to-Speech Pipeline Production Deployment: Best Practices for High-Performance Voice AI](/huggingface/speech-to-speech/best-practices-for-configuring-speech-to-speech-pipeline-for-production-deployment)

Deploy huggingface/speech-to-speech in production with best practices for high-performance voice AI. Learn to minimize latency using realtime mode GPU acceleration pooled pipelines and self-hosted LLM backends.

- Tags: best-practices
- Published: 2026-08-05

### [How to Integrate the Speech-to-Speech Pipeline with Hugging Face Inference Providers](/huggingface/speech-to-speech/how-to-integrate-speech-to-speech-pipeline-with-hugging-face-inference-providers)

Integrate the speech-to-speech pipeline with Hugging Face Inference Providers by setting the llm_backend and configuring your API key and base URL for seamless routing.

- Tags: how-to-guide
- Published: 2026-08-05

### [How to Manage Audio Format Conversions and Resampling in the Hugging Face Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-manage-audio-format-conversions-and-resampling-within-pipeline)

Learn to manage audio format conversions and resampling in the Hugging Face Speech-to-Speech pipeline. Discover how it handles real-time streams and model-generated audio efficiently.

- Tags: how-to-guide
- Published: 2026-08-05

### [How to Define and Use Custom Prompts for the LLM in the Hugging Face Speech‑to‑Speech Pipeline](/huggingface/speech-to-speech/how-to-define-and-use-custom-prompts-for-llm-in-pipeline)

Learn to define and use custom prompts for the LLM in the Hugging Face speech-to-speech pipeline. Build system prompts using build_text_system_prompt or build_voice_system_prompt and pass them to your handler.

- Tags: how-to-guide
- Published: 2026-08-05

### [How to Benchmark and Profile the Speech-to-Speech Pipeline in Hugging Face's S2S Library](/huggingface/speech-to-speech/how-to-benchmark-and-profile-performance-of-speech-to-speech-pipeline)

Benchmark and profile the Hugging Face speech-to-speech pipeline using built-in scripts and Python profilers. Measure end-to-end latency and analyze TTS/STT performance.

- Tags: performance
- Published: 2026-08-05

### [Hugging Face Speech-to-Speech LLM Proxy Feature: Complete Usage Guide](/huggingface/speech-to-speech/what-is-functionality-and-usage-of-llm-proxy-feature)

Learn how to use the Hugging Face Speech-to-Speech LLM proxy. This guide details how to forward requests to remote LLM backends with an OpenAI-compatible passthrough.

- Tags: how-to-guide
- Published: 2026-08-05

### [How to Handle Bidirectional Streaming Audio in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-handle-bidirectional-streaming-audio-in-speech-to-speech-pipeline)

Learn to handle bidirectional streaming audio in the Speech-to-Speech pipeline using Hugging Face. Discover the WebSocketStreamer, SpeechToSpeech pipeline, and utility modules for seamless audio processing.

- Tags: how-to-guide
- Published: 2026-08-05

### [How to Implement Voice Cloning Using Pocket TTS in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-implement-voice-cloning-using-pocket-tts-within-pipeline)

Learn to implement voice cloning with Pocket TTS in the Hugging Face Speech-to-Speech pipeline. Extract speaker embeddings from audio with PocketTTSHandler for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-05

### [How to Optimize TTS Performance for CUDA, Apple Silicon, and CPU: A Complete Guide](/huggingface/speech-to-speech/how-to-optimize-tts-performance-for-different-hardware-cuda-apple-silicon-cpu)

Optimize TTS performance across CUDA, Apple Silicon, and CPU. Discover automatic backend selection and tuning techniques like quantization and model choice for faster speech synthesis.

- Tags: performance
- Published: 2026-08-05

### [Speech-to-Speech TTS Backends Compared: 5 Options for Text-to-Speech Pipeline](/huggingface/speech-to-speech/what-are-key-differences-between-various-tts-backends-available)

Compare 5 speech-to-speech TTS backends like ChatTTS, MMS, Pocket TTS, Kokoro, and Qwen 3-TTS. Discover options for different hardware, languages, and voice customization needs.

- Tags: comparison
- Published: 2026-08-05

### [How to Implement Tool Calling in the Hugging Face Speech-to-Speech Voice Pipeline](/huggingface/speech-to-speech/how-to-implement-tool-calling-functionality-within-voice-pipeline)

Implement tool calling in Hugging Face speech-to-speech. Trigger external actions by defining tools, injecting prompts, and parsing code blocks into function calls.

- Tags: how-to-guide
- Published: 2026-08-05

### [How to Handle Audio Interruptions and Manage Turn-Taking in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-handle-audio-interruptions-and-manage-turn-taking-in-pipeline)

Learn to handle audio interruptions and manage turn-taking in the speech-to-speech pipeline. Discover the three-component architecture for seamless voice interaction.

- Tags: how-to-guide
- Published: 2026-08-05

### [How WebSocket Protocol Enables Real-Time Communication in the Hugging Face Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-does-websocket-protocol-enable-real-time-communication-in-speech-to-speech-pipeline)

Discover how WebSocket protocol powers real-time communication in the Hugging Face speech-to-speech pipeline, enabling low-latency audio streaming and instant text output.

- Tags: internals
- Published: 2026-08-05

### [How to Configure VAD Parameters for Specific Use Cases in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-configure-vad-parameters-for-specific-use-cases-in-speech-to-speech-pipeline)

Learn to configure VAD parameters for your speech-to-speech pipeline. Optimize silence, padding, and threshold for unique use cases using `VADHandlerArguments` or CLI flags.

- Tags: how-to-guide
- Published: 2026-08-05

### [How to Optimize the Speech‑to‑Speech Pipeline for Low Latency: Complete Configuration Guide](/huggingface/speech-to-speech/how-to-optimize-speech-to-speech-pipeline-for-low-latency)

Optimize your speech-to-speech pipeline for low latency. Learn to stream audio, use VAD thresholds, and configure fast TTS models for sub-500ms end-to-end performance.

- Tags: how-to-guide
- Published: 2026-08-05

### [How Multi-Language Support Works in the Hugging Face Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-is-multi-language-support-handled-in-speech-to-speech-pipeline)

Discover how the Hugging Face speech-to-speech pipeline masterfully handles multi-language support by seamlessly passing language codes through STT, LLM, and TTS components.

- Tags: internals
- Published: 2026-08-05

### [How to Run Speech-to-Speech Pipeline in 4 Modes: Realtime, Local, Socket, and Raw-WebSocket](/huggingface/speech-to-speech/different-modes-to-run-speech-to-speech-pipeline-realtime-local-socket-raw-websocket)

Explore four modes for the speech-to-speech pipeline: realtime, local, socket, and raw-websocket. Learn how to use diverse audio transport layers for your needs.

- Tags: how-to-guide
- Published: 2026-08-05

### [How the OpenAI Realtime API Integrates with the Speech‑to‑Speech Pipeline: Architecture and Code Guide](/huggingface/speech-to-speech/how-does-openai-realtime-api-integrate-with-speech-to-speech-pipeline)

Learn how the OpenAI Realtime API integrates with the speech-to-speech pipeline. Discover the architecture and code for this WebSocket bridge, synchronizing VAD, STT, LLM, and TTS components for seamless audio processing.

- Tags: architecture
- Published: 2026-08-05

### [LLM Backends Supported by the Hugging Face Speech-to-Speech Pipeline: Complete Configuration Guide](/huggingface/speech-to-speech/what-llm-backends-are-supported-by-speech-to-speech-pipeline-and-how-to-configure-them)

Discover LLM backends for Hugging Face speech-to-speech: transformers, mlx-lm, responses-api, and chat-completions. Get your complete configuration guide here.

- Tags: how-to-guide
- Published: 2026-08-05

### [How to Configure and Use Different Text‑to‑Speech (TTS) Backends in Hugging Face Speech‑to‑Speech](/huggingface/speech-to-speech/how-to-configure-and-use-different-text-to-speech-tts-backends-for-speech-to-speech)

Configure diverse Text-to-Speech TTS backends MLX GGML or Torch within Hugging Face Speech-to-Speech with simple argument settings. Enhance your voice generation now.

- Tags: how-to-guide
- Published: 2026-08-05

### [How to Configure and Swap Speech-to-Text (STT) Backends in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-configure-and-swap-speech-to-text-stt-backends-in-speech-to-speech-pipeline)

Easily configure and swap Speech-to-Text STT backends in the Hugging Face speech-to-speech pipeline. Learn how to set the stt field in ModuleArguments for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-05

### [How Voice Activity Detection (VAD) Works in the HuggingFace Speech-to-Speech Pipeline: A Deep Architecture Guide](/huggingface/speech-to-speech/how-does-voice-activity-detection-vad-work-in-huggingface-speech-to-speech-pipeline)

Explore Voice Activity Detection VAD within the HuggingFace speech-to-speech pipeline. Learn how its two-stage architecture optimizes audio processing for accurate speech event triggering.

- Tags: deep-dive
- Published: 2026-08-05

### [How the LLM Proxy Enables Concurrent Text Requests in Speech-to-Speech](/huggingface/speech-to-speech/how-llm-proxy-feature-enables-concurrent-text-requests)

Discover how the LLM proxy in huggingface/speech-to-speech enables concurrent text requests. This FastAPI passthrough forwards requests asynchronously, allowing parallel text and audio processing without resource contention.

- Tags: internals
- Published: 2026-08-04

### [How to Add a Custom STT Backend to the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-add-custom-stt-backend-to-pipeline)

Learn how to add a custom STT backend to the speech-to-speech pipeline. Inherit from BaseSTTHandler, create a config, register your STT backend, and wire it into the pipeline. Extend your speech processing capabilities today!

- Tags: how-to-guide
- Published: 2026-08-04

### [How the Hugging Face Speech-to-Speech Pipeline Handles Concurrent WebSocket Sessions](/huggingface/speech-to-speech/how-pipeline-handles-concurrent-websocket-sessions)

Learn how the Hugging Face Speech-to-Speech pipeline efficiently handles concurrent WebSocket sessions using isolated PipelineUnit instances and private queues for zero cross-talk.

- Tags: internals
- Published: 2026-08-04

### [Rate Limiting and Authentication for Production Deployment in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/rate-limiting-authentication-production-deployment)

Secure your Hugging Face Speech-to-Speech deployment with built-in stateless authentication and tier-based rate limiting. Protect your service and manage usage effectively.

- Tags: best-practices
- Published: 2026-08-03

### [Connecting Vision-Language Models via mlx-lm on Apple Silicon: Implementation Guide](/huggingface/speech-to-speech/connecting-vision-language-models-mlx-lm-apple-silicon)

Learn how to connect vision-language models using mlx-lm on Apple Silicon. This guide details low-latency inference with Metal Performance Shaders for efficient multimodal processing.

- Tags: implementation-guide
- Published: 2026-08-03

### [Implementing Custom Function Tools for Robot Control with Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/implementing-custom-function-tools-robot-control)

Extend Hugging Face Speech-to-Speech with custom function tools. Define JSON schemas, register via OpenAI API, and implement execution hooks for robot control.

- Tags: how-to-guide
- Published: 2026-08-03

### [CUDA Version Compatibility for Qwen3-TTS on Linux: Requirements and Setup Guide](/huggingface/speech-to-speech/cuda-version-compatibility-qwen3-tts-linux)

Learn Qwen3-TTS CUDA version compatibility for Linux. Ensure optimal performance with CUDA 11.8 or 12.0+. Avoid CPU fallback. Get the setup guide now.

- Tags: how-to-guide
- Published: 2026-08-03

### [Memory Optimization for Running Multiple Pipeline Instances in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/memory-optimization-running-multiple-pipeline-instances)

Optimize memory for multiple Hugging Face speech-to-speech pipeline instances by sharing models, using lightweight backends, and bounding queues to avoid GPU/CPU exhaustion.

- Tags: performance
- Published: 2026-08-03

### [DeepFilterNet Audio Enhancement Conflicts with Pocket TTS in Speech-to-Speech Pipeline](/huggingface/speech-to-speech/deepfilternet-audio-enhancement-conflicts-pocket-tts)

Resolve DeepFilterNet audio enhancement conflicts with Pocket TTS in your speech-to-speech pipeline. Learn why these components clash and how to fix them for seamless audio processing.

- Tags: deep-dive
- Published: 2026-08-03

### [Response Cancellation and Barge-In in Hugging Face Speech-to-Speech: A Complete Guide](/huggingface/speech-to-speech/response-cancellation-barge-in-user-interrupts)

Learn how Hugging Face speech-to-speech handles user interruptions with CancelScope for response cancellation and barge-in. Detects SpeechStartedEvent to abort generation.

- Tags: how-to-guide
- Published: 2026-08-03

### [How to Configure Session Instructions, Tools, and Voice Settings at Runtime in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/session-configuration-instructions-tools-voice-settings-runtime)

Configure Hugging Face speech-to-speech runtime session instructions tools and voice settings dynamically using typed dataclass arguments and ParsedArguments for flexible STT LLM and TTS handler assembly.

- Tags: how-to-guide
- Published: 2026-08-03

### [Speech-to-Speech Threading Model: How VAD, STT, LLM, and TTS Communicate via Queues](/huggingface/speech-to-speech/threading-model-vad-stt-llm-tts-communication-queues)

Explore the speech-to-speech threading model. Discover how VAD, STT, LLM, and TTS communicate efficiently using thread-safe queues in this producer-consumer architecture.

- Tags: internals
- Published: 2026-08-03

### [How to Debug Audio Pipeline Latency Issues End-to-End in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/debugging-audio-pipeline-latency-issues-end-to-end)

Debug audio pipeline latency end-to-end in Hugging Face speech-to-speech. Tune VAD chunk sizes, enable live transcription, and select streaming TTS modes for lower latency.

- Tags: how-to-guide
- Published: 2026-08-03

### [Qwen3-TTS GGML vs MLX Backend Configuration on Different Platforms: A Complete Guide](/huggingface/speech-to-speech/qwen3-tts-ggml-vs-mlx-backend-configuration-platforms)

Configure Qwen3-TTS GGML vs MLX backends on Apple Silicon Linux and Windows. Optimize your speech-to-speech setup with this comprehensive guide.

- Tags: how-to-guide
- Published: 2026-08-03

### [LLM Proxy: Exposing the Configured LLM as an OpenAI-Compatible Endpoint in Hugging Face speech-to-speech](/huggingface/speech-to-speech/llm-proxy-exposing-llm-as-openai-compatible-endpoint)

Use the LLM Proxy in Hugging Face speech-to-speech to expose your configured LLM as an OpenAI-compatible API. This FastAPI proxy seamlessly forwards requests to your upstream service.

- Tags: how-to-guide
- Published: 2026-08-03

### [WebRTC Transport Setup with STUN/TURN Servers for Production Deployment in Speech-to-Speech](/huggingface/speech-to-speech/webrtc-transport-setup-stun-turn-servers-production-deployment)

Set up WebRTC transport with STUN/TURN servers for production speech-to-speech. Configure ICE servers via environment variables for robust peer-to-peer connections.

- Tags: how-to-guide
- Published: 2026-08-03

### [Configuring Multi-Language Detection and Switching in HuggingFace Speech-to-Speech](/huggingface/speech-to-speech/configuring-multi-language-detection-and-switching-languages)

Enable multi language detection and switching for HuggingFace speech-to-speech. Set language to auto in STT and TTS handlers for seamless multilingual conversations.

- Tags: how-to-guide
- Published: 2026-08-03

### [How to Enable Live Transcription During Voice Conversations with Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/enabling-live-transcription-during-voice-conversations)

Enable live transcription in Hugging Face Speech-to-Speech. Set enable_live_transcription=True to stream partial transcriptions in real-time during voice conversations.

- Tags: how-to-guide
- Published: 2026-08-03

### [Using the OpenAI Realtime WebSocket Protocol with Custom Clients](/huggingface/speech-to-speech/using-openai-realtime-websocket-protocol-with-custom-clients)

Integrate OpenAI Realtime WebSocket with custom clients using huggingface/speech-to-speech. Connect via standard JSON events for STT, LLM, and TTS processing.

- Tags: how-to-guide
- Published: 2026-08-03

### [Swapping STT Backends: Parakeet TDT vs Whisper vs Faster Whisper in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/swapping-stt-backends-parakeet-tdt-vs-whisper-vs-faster-whisper)

Easily swap STT backends like Parakeet TDT, Whisper, and Faster Whisper in Hugging Face Speech-to-Speech. Discover flexible speech processing with a single flag.

- Tags: comparison
- Published: 2026-08-03

### [How to Configure Silero VAD Thresholds for Different Acoustic Environments](/huggingface/speech-to-speech/configuring-silero-vad-thresholds-for-different-acoustic-environments)

Learn to configure Silero VAD thresholds for noisy or quiet environments. Adjust the threshold when initializing VADIterator for optimal speech detection.

- Tags: how-to-guide
- Published: 2026-08-03

### [How the VAD Pipeline Handles Turn-Taking and Interruption Detection in Speech-to-Speech](/huggingface/speech-to-speech/how-vad-pipeline-handles-turn-taking-and-interruption-detection)

Discover how the VAD pipeline manages turn-taking and interruption detection in speech-to-speech conversations. Learn about VADHandler, SpeculativeTurnTracker, and RuntimeConfig.

- Tags: internals
- Published: 2026-08-03

### [How to Use Different Vocoders with Speech-to-Speech Models in Hugging Face's speech-to-speech Library](/huggingface/speech-to-speech/how-to-use-different-vocoders-speech-to-speech-models)

Learn to swap vocoders in Hugging Face speech-to-speech models. Configure Qwen3TTSHandlerArguments with ggml or torch backends for custom talker and codec components.

- Tags: how-to-guide
- Published: 2026-08-02

### [Limitations of Current Speech-to-Speech Models: A Deep Dive into the Hugging Face Pipeline](/huggingface/speech-to-speech/what-are-limitations-current-speech-to-speech-models)

Explore current speech-to-speech model limitations. Understand cascading latency, fragmented coverage, hardware issues, and architectural brittleness in Hugging Face pipelines.

- Tags: deep-dive
- Published: 2026-08-02

### [How to Handle Real-Time Speech-to-Speech Processing with the Hugging Face speech-to-speech Library](/huggingface/speech-to-speech/how-to-handle-real-time-speech-to-speech-processing)

Master real-time speech-to-speech processing with the Hugging Face library. Learn to build low-latency streaming pipelines for seamless voice interaction.

- Tags: how-to-guide
- Published: 2026-08-02

### [How to Contribute to the HuggingFace Speech-to-Speech Project: A Complete Guide](/huggingface/speech-to-speech/how-to-contribute-huggingface-speech-to-speech-project)

Learn how to contribute to the HuggingFace speech-to-speech project. Follow our guide to set up your environment, understand the pipeline, and submit pull requests to enhance this open-source initiative.

- Tags: how-to-guide
- Published: 2026-08-02

### [How the Transformers Library Powers the Hugging Face Speech-to-Speech Project](/huggingface/speech-to-speech/what-role-transformers-library-project)

Discover how the transformers library powers Hugging Face speech-to-speech. It handles model loading, tokenization, and generation for LLM, TTS, and STT.

- Tags: deep-dive
- Published: 2026-08-02

### [How to Deal with Noise in Input Audio: A Complete Guide for the Hugging Face Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-deal-with-noise-input-audio)

Learn how to deal with noise in input audio for the Hugging Face Speech-to-Speech pipeline. Discover VAD handler techniques including noise floor threshold, buffering, and DeepFilterNet enhancement.

- Tags: how-to-guide
- Published: 2026-08-02

### [Recommended Hardware for Training and Inference in Speech-to-Speech Pipelines](/huggingface/speech-to-speech/what-hardware-recommended-training-inference)

Discover recommended hardware for speech-to-speech inference. Optimize performance with NVIDIA CUDA GPUs for low latency or choose Apple Silicon M1/M2/M3 for native speed.

- Tags: performance
- Published: 2026-08-02

### [How to Deploy a Speech-to-Speech Model with the Hugging Face Speech-to-Speech Repository](/huggingface/speech-to-speech/how-to-deploy-speech-to-speech-model)

Deploy a speech-to-speech model easily with the Hugging Face repository. Select your backends and run the OpenAI Realtime-compatible WebSocket server for seamless integration.

- Tags: how-to-guide
- Published: 2026-08-02

### [Ethical Considerations for Speech-to-Speech Technology: A Developer's Guide to Responsible Implementation](/huggingface/speech-to-speech/what-are-ethical-considerations-speech-to-speech-technology)

Learn the ethical considerations for speech-to-speech technology. Developers explore privacy, bias, voice cloning, and safety guardrails for responsible implementation.

- Tags: best-practices
- Published: 2026-08-02

### [How to Integrate Hugging Face Speech‑to‑Speech into a Python Application](/huggingface/speech-to-speech/how-to-integrate-huggingface-speech-to-speech-python-application)

Integrate Hugging Face speech-to-speech into your Python app. Learn to build and run a voice agent pipeline including VAD, STT, LLM, and TTS. Get started today.

- Tags: how-to-guide
- Published: 2026-08-02

### [Speech-to-Speech Translation Challenges: Technical Hurdles in Real-Time Voice AI](/huggingface/speech-to-speech/what-are-challenges-speech-to-speech-translation)

Explore speech to speech translation challenges. Discover technical hurdles in real time voice AI including latency, multilingual models, and hardware diversity. Learn more at Hugging Face.

- Tags: deep-dive
- Published: 2026-08-02

### [How to Evaluate the Performance of a Speech-to-Speech Model](/huggingface/speech-to-speech/how-to-evaluate-performance-speech-to-speech-model)

Evaluate speech-to-speech model performance with huggingface tools. Measure latency, real-time factor, and TTS/STT component speed for better results.

- Tags: performance
- Published: 2026-08-02

### [Can I Use Custom Datasets for Training Speech-to-Speech Models? A Deep Dive into the Hugging Face Repository](/huggingface/speech-to-speech/can-i-use-custom-datasets-training-speech-to-speech-models)

Cannot train speech-to-speech models on custom datasets with huggingface/speech-to-speech. This repository is for inference only, lacking a training pipeline. Learn more about its limitations.

- Tags: deep-dive
- Published: 2026-08-02

### [How to Handle Different Audio Sampling Rates in the Hugging Face Speech‑to‑Speech Pipeline](/huggingface/speech-to-speech/how-to-handle-different-audio-sampling-rates)

Learn how to handle different audio sampling rates in the Hugging Face speech-to-speech pipeline. Discover automatic resampling techniques and supported libraries for seamless audio processing.

- Tags: how-to-guide
- Published: 2026-08-02

### [Speech-to-Speech Input and Output Formats: Complete Audio Specification for the Hugging Face Realtime API](/huggingface/speech-to-speech/what-input-output-formats-speech-to-speech-models)

Discover the complete audio specification for Hugging Face speech-to-speech models. Learn about input and output formats: 16 kHz PCM, mono, 16-bit signed, base64-encoded chunks.

- Tags: api-reference
- Published: 2026-08-02

### [How to Perform Voice Conversion with HuggingFace Speech-to-Speech: A Complete Guide](/huggingface/speech-to-speech/how-to-perform-voice-conversion-huggingface-speech-to-speech)

Learn to perform voice conversion with HuggingFace speech-to-speech by cloning audio or using speaker embeddings for Qwen3-TTS. Get started with our complete guide today.

- Tags: how-to-guide
- Published: 2026-08-02

### [How to Convert Speech from One Language to Another with Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-to-convert-speech-one-language-another-library)

Easily convert speech to another language using Hugging Face speech-to-speech. Learn the three-stage pipeline STT LM TTS for seamless voice translation.

- Tags: how-to-guide
- Published: 2026-08-02

### [Speech-to-Speech Model Architecture in the Hugging Face Speech-to-Speech Repository](/huggingface/speech-to-speech/what-is-architecture-speech-to-speech-models)

Explore the modular four-stage speech-to-speech model architecture in the Hugging Face repository. Discover how VAD STT LLM TTS stages enable low-latency voice assistants.

- Tags: architecture
- Published: 2026-08-02

### [Pre-Trained Models for Speech-to-Speech: Complete Guide to the Hugging Face Pipeline](/huggingface/speech-to-speech/what-pre-trained-models-available-speech-to-speech)

Explore pre-trained models for speech-to-speech with the Hugging Face pipeline. Discover Silero VAD, Parakeet TDT, OpenAI GPT, and Qwen3-TTS for your audio projects.

- Tags: tutorial
- Published: 2026-08-02

### [How to Run a Basic Speech-to-Speech Example with HuggingFace: Complete Setup Guide](/huggingface/speech-to-speech/how-to-run-basic-speech-to-speech-example-huggingface)

Learn how to run a basic speech-to-speech example with HuggingFace. Follow our complete setup guide to get your real-time pipeline running quickly.

- Tags: getting-started
- Published: 2026-08-02

### [Dependencies for Hugging Face Speech-to-Speech: Complete Installation Guide](/huggingface/speech-to-speech/what-are-the-dependencies-for-huggingface-speech-to-speech)

Install Hugging Face speech-to-speech effortlessly. Discover core dependencies like FastAPI and PyTorch, plus platform-specific ML libraries for macOS and optional extras for advanced TTS/STT.

- Tags: getting-started
- Published: 2026-08-02

### [Limitations of Speech-to-Speech Models: Constraints in the Hugging Face Pipeline](/huggingface/speech-to-speech/limitations-current-speech-to-speech-models)

Explore limitations of speech-to-speech models like restricted language support, hardware issues, LLM latency, and dependency conflicts in the Hugging Face pipeline. Improve your understanding and find solutions.

- Tags: deep-dive
- Published: 2026-08-01

### [How to Use the speech-to-speech Python API: A Complete Guide](/huggingface/speech-to-speech/how-to-use-speech-to-speech-python-api)

Learn how to use the speech-to-speech Python API to build modular voice assistants. This guide covers VAD, STT, LLM, and TTS components with the ThreadManager.

- Tags: how-to-guide
- Published: 2026-08-01

### [Speech-to-Speech vs Text-to-Speech: Understanding the Key Difference in the Hugging Face Pipeline](/huggingface/speech-to-speech/difference-speech-to-speech-vs-text-to-speech)

Learn the key difference between speech-to-speech and text-to-speech in the Hugging Face pipeline. Speech-to-speech is a full voice pipeline, while text-to-speech is just the final synthesis step.

- Tags: deep-dive
- Published: 2026-08-01

### [How to Perform Voice Conversion with Speech-to-Speech: A Complete Pipeline Guide](/huggingface/speech-to-speech/perform-voice-conversion-speech-to-speech)

Master voice conversion with speech-to-speech. Learn the four-stage pipeline VAD STT LLM TTS for real-time voice transformation and synthesize any target voice.

- Tags: tutorial
- Published: 2026-08-01

### [Recommended Hardware for Optimal Speech-to-Speech Performance: GPU Requirements and Setup Guide](/huggingface/speech-to-speech/hardware-recommended-optimal-speech-to-speech-performance)

Unlock optimal speech-to-speech performance with our GPU requirements guide. Learn the recommended hardware setup and VRAM needs for accelerated LLM processing.

- Tags: how-to-guide
- Published: 2026-08-01

### [How to Run Speech-to-Speech on a GPU: Complete Setup Guide](/huggingface/speech-to-speech/run-speech-to-speech-on-gpu)

Accelerate speech-to-speech with GPU acceleration. Follow our complete setup guide to install Qwen3-TTS, configure PyTorch for CUDA, and run inference on your GPU for faster results.

- Tags: how-to-guide
- Published: 2026-08-01

### [Are There Pre-Trained Speech-to-Speech Models Available?](/huggingface/speech-to-speech/pre-trained-speech-to-speech-models-available)

Discover readily available pre-trained speech-to-speech models from huggingface. Access and utilize multiple models directly with automatic downloads from the Hugging Face Hub.

- Tags: getting-started
- Published: 2026-08-01

### [Architecture of the Default Speech-to-Speech Models in Hugging Face's Pipeline](/huggingface/speech-to-speech/architecture-default-speech-to-speech-models)

Explore the default speech-to-speech architecture in Hugging Face's pipeline. Understand how VAD, TDT, LLM, and TTS handlers create a real-time speech conversion system.

- Tags: architecture
- Published: 2026-08-01

### [How to Contribute to the HuggingFace Speech-to-Speech Project: A Complete Developer Guide](/huggingface/speech-to-speech/contribute-huggingface-speech-to-speech-project)

Learn how to contribute to the HuggingFace Speech-to-Speech project. Fork the repo, set up your environment, implement changes, run tests, and submit a PR. Your guide to impactful contributions.

- Tags: how-to-guide
- Published: 2026-08-01

### [Licensing Details for the Speech-to-Speech Models: Repository Code vs. Model Weights](/huggingface/speech-to-speech/licensing-details-speech-to-speech-models)

Understand the licensing for speech-to-speech models. Explore the Apache 2.0 repo license and individual model licenses for Parler TTS, Melo TTS, and more.

- Tags: licensing-details
- Published: 2026-08-01

### [How to Prepare Data for Speech-to-Speech Training: A Complete Guide](/huggingface/speech-to-speech/prepare-data-speech-to-speech-training)

Learn how to prepare data for speech-to-speech training. Provide 16 kHz mono audio with UTF-8 transcripts organized as a Hugging Face Dataset for optimal model performance.

- Tags: how-to-guide
- Published: 2026-08-01

### [Dependencies for Speech-to-Speech: Complete Guide to Installation Requirements](/huggingface/speech-to-speech/dependencies-for-speech-to-speech)

Discover the essential dependencies for speech-to-speech installation. Learn about core runtime requirements and optional extras for specific model backends.

- Tags: how-to-guide
- Published: 2026-08-01

### [How to Evaluate the Performance of a Speech-to-Speech Model: A Complete Benchmarking Guide](/huggingface/speech-to-speech/evaluate-performance-speech-to-speech-model)

Learn how to evaluate speech-to-speech model performance with Hugging Face's benchmark scripts. Measure latency, real-time factor, and more across the entire pipeline.

- Tags: tutorial
- Published: 2026-08-01

### [Common Speech-to-Speech Errors and Troubleshooting Tips for the Hugging Face Pipeline](/huggingface/speech-to-speech/speech-to-speech-errors-troubleshooting)

Resolve common speech-to-speech errors with our expert troubleshooting guide. Fix platform-device mismatches, dependency issues, and backend configurations for seamless Hugging Face pipeline performance.

- Tags: how-to-guide
- Published: 2026-08-01

### [How to Integrate Speech-to-Speech into a Python Project: A Complete Developer Guide](/huggingface/speech-to-speech/integrate-speech-to-speech-python-project)

Integrate speech-to-speech into your Python project with Hugging Face. Follow this developer guide to build a four-stage pipeline for seamless voice interaction.

- Tags: how-to-guide
- Published: 2026-08-01

### [Speech-to-Speech Tasks Supported by the Hugging Face speech-to-speech Repository](/huggingface/speech-to-speech/different-speech-to-speech-tasks-supported)

Explore speech-to-speech tasks like VAD STT LLM generation and TTS supported by Hugging Face. Mix and match pluggable backends easily via CLI for custom solutions.

- Tags: getting-started
- Published: 2026-08-01

### [Using Speech-to-Speech for Real-Time Applications: Architecture and Setup Guide](/huggingface/speech-to-speech/speech-to-speech-real-time-applications)

Explore speech-to-speech for real-time applications. This guide details the architecture and setup for sub-second latency streaming using VAD, STT, LLM, and TTS components.

- Tags: how-to-guide
- Published: 2026-08-01

### [How to Convert Speech from One Language to Another Using the Hugging Face Speech-to-Speech Library](/huggingface/speech-to-speech/how-to-translate-speech-language-conversion)

Learn how to convert speech from one language to another using the Hugging Face speech-to-speech library. Configure the S2SPipeline for seamless language translation and synthesis.

- Tags: how-to-guide
- Published: 2026-08-01

### [How to Fine-Tune a Speech-to-Speech Model: A Complete Guide for the Hugging Face Pipeline](/huggingface/speech-to-speech/how-to-fine-tune-speech-to-speech-model)

Learn to fine-tune a speech-to-speech model using the Hugging Face pipeline. Independently train STT LLM or TTS components and integrate them for custom speech applications.

- Tags: how-to-guide
- Published: 2026-08-01

### [What Models Are Supported by Hugging Face Speech-to-Speech: Complete Model Guide](/huggingface/speech-to-speech/what-models-are-supported-by-huggingface-speech-to-speech)

Explore Hugging Face Speech-to-Speech models for voice detection, STT, LLMs, and TTS. Discover supported options like Whisper, OpenAI APIs, Qwen3-TTS, and Kokoro.

- Tags: api-reference
- Published: 2026-08-01

### [How to Use the Speech-to-Speech Command-Line Interface: A Complete Guide to huggingface/speech-to-speech](/huggingface/speech-to-speech/how-to-use-speech-to-speech-cli)

Master the speech-to-speech command-line interface. Learn to configure VAD, STT, LLM, and TTS components for a custom audio pipeline. Get the complete guide.

- Tags: how-to-guide
- Published: 2026-08-01

### [Speech-to-Speech System Requirements: Complete Hardware & Software Guide](/huggingface/speech-to-speech/what-are-the-system-requirements-for-speech-to-speech)

Discover Hugging Face speech-to-speech system requirements. Learn about Python 3.10+, CPU/GPU compatibility, and automatic CUDA/Apple Silicon backend selection for your platform.

- Tags: getting-started
- Published: 2026-08-01

### [How to Install Hugging Face Speech-to-Speech: Complete Setup Guide](/huggingface/speech-to-speech/how-to-install-huggingface-speech-to-speech)

Install Hugging Face Speech-to-Speech easily with pip. Get the complete VAD STT LLM TTS pipeline for Linux CUDA, macOS MLX, or CPU. Follow our setup guide.

- Tags: getting-started
- Published: 2026-08-01

### [How to Configure ICE Servers for WebRTC Deployment Behind NAT](/huggingface/speech-to-speech/how-to-configure-ice-servers-webrtc-deployment-behind-nat)

Configure ICE servers for WebRTC behind NAT. Learn how to set the SPEECH_TO_SPEECH_ICE_SERVERS environment variable for seamless STUN/TURN server integration in your deployment.

- Tags: how-to-guide
- Published: 2026-07-31

### [LMOutputProcessor: How It Routes LLM Output to TTS and Text Events in Speech-to-Speech](/huggingface/speech-to-speech/what-is-lmoutputprocessor-how-it-splits-output-tts-text-events)

Discover how LMOutputProcessor routes LLM output to TTS and text events. This component ensures real-time sync between displayed text and generated speech in speech-to-speech.

- Tags: internals
- Published: 2026-07-31

### [How TranscriptionNotifier Taps Into Transcripts for Events in Speech-to-Speech Pipelines](/huggingface/speech-to-speech/how-transcriptionnotifier-taps-into-transcripts-events)

Discover how TranscriptionNotifier uses speech-to-text output to create events for WebSocket routing and optional LLM processing in speech-to-speech pipelines.

- Tags: internals
- Published: 2026-07-31

### [How the Multi-Pipeline Pool (`--num_pipelines`) Enables Concurrent Sessions in Speech-to-Speech](/huggingface/speech-to-speech/how-multi-pipeline-pool-num-pipelines-works-concurrent-sessions)

Learn how the multi-pipeline pool (--num_pipelines) enables concurrent sessions in Hugging Face speech-to-speech by using isolated pipeline instances for efficient processing.

- Tags: internals
- Published: 2026-07-31

### [Understanding RuntimeConfig and session.update in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/what-is-runtimeconfig-how-session-update-modifies-pipeline-behavior)

Learn how RuntimeConfig and session.update in Hugging Face Speech-to-Speech dynamically control VAD, LLM, and TTS behavior without pipeline restarts. Update conversation state and session parameters easily.

- Tags: deep-dive
- Published: 2026-07-31

### [How response.cancel Interrupts Generation in the Speech-to-Speech Realtime Engine](/huggingface/speech-to-speech/how-response-cancel-mechanism-works-interrupting-generation)

Understand how response.cancel interrupts generation in the huggingface speech-to-speech engine. Learn how it aborts LLM and TTS generation for instant system reset.

- Tags: internals
- Published: 2026-07-31

### [Thread Model for the Four-Stage VAD → STT → LLM → TTS Pipeline in Speech-to-Speech](/huggingface/speech-to-speech/thread-model-four-stage-pipeline-vad-stt-llm-tts)

Explore the thread model for the four-stage VAD STT LLM TTS speech-to-speech pipeline. Learn how asynchronous coroutines ensure real-time performance without thread contention.

- Tags: internals
- Published: 2026-07-31

### [How to Enable and Configure Live Transcription Events in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-to-enable-configure-live-transcription-events)

Learn to enable and configure live transcription events in Hugging Face Speech-to-Speech. Set enable_live_transcription=True and adjust the update interval for real-time audio processing.

- Tags: how-to-guide
- Published: 2026-07-31

### [GGML MLX vs Torch Backend Performance Comparison for Qwen3-TTS](/huggingface/speech-to-speech/performance-differences-ggml-mlx-torch-backends-qwen3-tts)

Discover GGML MLX vs Torch backend performance for Qwen3-TTS. GGML MLX offers 2-3x faster inference, lower latency, and less memory use than Torch. Learn more about these speed improvements.

- Tags: performance
- Published: 2026-07-31

### [WebRTC Transport Architecture in Speech-to-Speech: How It Differs from WebSocket Mode](/huggingface/speech-to-speech/webrtc-transport-architecture-vs-websocket-mode)

Explore the WebRTC transport architecture for speech-to-speech, differentiating it from WebSocket mode by understanding RTP packet streaming and data channel routing for efficient audio transmission.

- Tags: architecture
- Published: 2026-07-31

### [How the Speech-to-Speech VAD Pipeline Handles Turn-Taking and Interruption Detection with Silero VAD](/huggingface/speech-to-speech/how-vad-pipeline-handles-turn-taking-interruption-detection-silero-vad)

Discover how the Silero VAD pipeline expertly manages turn-taking and interruption detection. Learn about state-machine VAD handling for seamless speech boundary detection and dynamic turn reopening.

- Tags: deep-dive
- Published: 2026-07-31

### [LLM Reasoning Tokens in Speech-to-Speech: How They Work and How to Disable Them](/huggingface/speech-to-speech/llm-reasoning-tokens-generation-disable)

Understand how LLM reasoning tokens impact speech-to-speech and learn to disable them using the disable thinking flag or reasoning effort argument in your OpenAI-compatible backend.

- Tags: internals
- Published: 2026-07-30

### [Qwen3-TTS vs Pocket TTS vs ChatTTS vs MMS TTS: Voice Quality and Latency Comparison](/huggingface/speech-to-speech/tts-backends-comparison-qwen3-pocket-chattts-mms-quality-latency)

Compare Qwen3-TTS, Pocket TTS, ChatTTS, and MMS TTS. Discover voice quality and latency differences to choose the best TTS for your needs. See which offers low latency and high fidelity.

- Tags: comparison
- Published: 2026-07-30

### [How the CancelScope Mechanism Enables Graceful Shutdown in Speech-to-Speech Pipelines](/huggingface/speech-to-speech/cancelscope-mechanism-graceful-shutdown-cancellation)

Learn how the CancelScope mechanism in huggingface/speech-to-speech ensures graceful shutdown and cancellation. Prevent race conditions and stale output with this thread-safe primitive.

- Tags: internals
- Published: 2026-07-30

### [How to Configure Logging and Debugging for the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/pipeline-logging-debugging-options-troubleshooting)

Troubleshoot the speech-to-speech pipeline with Hugging Face. Learn about CLI verbosity, handler timing, pipeline prefixes, and torch-compile diagnostics using helpful logging and debugging tools.

- Tags: how-to-guide
- Published: 2026-07-30

### [How to Configure Kokoro TTS with Voice Presets and Custom Voice Files](/huggingface/speech-to-speech/configure-kokoro-tts-voice-presets-custom-files)

Learn to configure Kokoro TTS with voice presets. Discover how to manage voice settings at handler, runtime, and LLM metadata levels for dynamic speech synthesis.

- Tags: how-to-guide
- Published: 2026-07-30

### [Speech-to-Speech Pipeline Audio Formats and Sample Rates: Complete Input/Output Guide](/huggingface/speech-to-speech/pipeline-audio-formats-sample-rates-input-output)

Learn the Hugging Face speech-to-speech pipeline's audio format requirements. Understand the expected 16-bit int16 mono audio at 16 kHz for seamless input and output.

- Tags: api-reference
- Published: 2026-07-30

### [How LMOutputProcessor Streams LLM Text to the TTS Pipeline](/huggingface/speech-to-speech/lmoutputprocessor-streaming-text-llm-tts-synthesis)

Discover how LMOutputProcessor streams LLM text to TTS, forwarding chunks via WebSocket and routing audio-eligible text for synthesis. Learn about its dual-channel handling.

- Tags: internals
- Published: 2026-07-30

### [MLX Lock Contention on Apple Silicon: How `--num_pipelines` Affects Realtime Performance in Speech-to-Speech](/huggingface/speech-to-speech/mlx-lock-contention-apple-silicon-num-pipelines)

Understand MLX lock contention on Apple Silicon impacting speech-to-speech realtime performance. Learn how --num_pipelines affects GPU access and prevents timeouts.

- Tags: performance
- Published: 2026-07-30

### [How to Add a Custom STT, LLM, or TTS Handler to the Speech-to-Speech Pipeline Architecture](/huggingface/speech-to-speech/add-custom-stt-llm-tts-handler-pipeline)

Integrate custom STT, LLM, or TTS handlers into the Hugging Face speech-to-speech pipeline. Learn to subclass BaseHandler, implement methods, and register your custom components.

- Tags: how-to-guide
- Published: 2026-07-30

### [Comparing STT Backends in Hugging Face Speech-to-Speech: Parakeet TDT, Whisper, and Alternatives](/huggingface/speech-to-speech/stt-backends-comparison-parakeet-whisper-faster-whisper)

Compare STT backends in Hugging Face Speech-to-Speech: Parakeet TDT, Whisper, Faster Whisper, and MLX-Audio. Discover hardware needs, language support, and latency for your ideal model.

- Tags: comparison
- Published: 2026-07-30

### [How the Speech-to-Speech Pipeline Handles Interruption When the User Speaks While AI Is Responding](/huggingface/speech-to-speech/pipeline-interruption-handling-user-speaking-ai-responding)

Learn how the speech-to-speech pipeline handles user interruption by canceling active AI streams and processing new input. Discover efficient real-time interaction.

- Tags: internals
- Published: 2026-07-30

### [Multi-Language Support and Automatic Language Detection in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/speech-to-speech-multi-language-support-automatic-detection)

Explore multi-language support and automatic language detection in Hugging Face Speech-to-Speech. Learn how Whisper and Parakeet TDT identify languages.

- Tags: how-to-guide
- Published: 2026-07-30

### [How Tool Calls Work Through the LLM Integration in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/llm-tool-calls-integration-events)

Understand LLM tool calls in Hugging Face Speech-to-Speech. Learn how tool calls are processed, validated, and emit OpenAI Realtime compatible function_call events via WebSocket.

- Tags: deep-dive
- Published: 2026-07-30

### [Differences Between responses-api and chat-completions LLM Backends in Speech-to-Speech](/huggingface/speech-to-speech/llm-backends-responses-api-vs-chat-completions-comparison)

Understand responses-api vs chat-completions LLM backends. Choose the right one for basic streaming or advanced tool-call streaming and reasoning controls.

- Tags: deep-dive
- Published: 2026-07-30

### [How to Configure the Qwen3-TTS Backend: GGML vs Torch on Linux, Windows, and macOS](/huggingface/speech-to-speech/configure-qwen3-tts-ggml-vs-torch-backends)

Master the Qwen3-TTS backend! Learn to configure GGML vs Torch on Linux, Windows, and macOS. Optimize your speech-to-speech setup easily.

- Tags: how-to-guide
- Published: 2026-07-30

### [How Live Transcription Works with Parakeet TDT on Apple Silicon](/huggingface/speech-to-speech/live-transcription-parakeet-tdt-apple-silicon-performance)

Discover how live transcription with Parakeet TDT on Apple Silicon achieves low latency using the MLX framework and GPU acceleration for seamless real-time speech-to-text.

- Tags: how-to-guide
- Published: 2026-07-30

### [Huggingface Speech-to-Speech VAD Turn-Taking Parameters: Configuring --thresh, --min_speech_ms, and --min_silence_ms](/huggingface/speech-to-speech/vad-parameters-turn-taking-behavior-threshold-min-speech-min-silence)

Discover how to control Huggingface Speech-to-Speech VAD turn-taking with --thresh, --min_speech_ms, and --min_silence_ms parameters. Optimize your audio processing.

- Tags: how-to-guide
- Published: 2026-07-30

### [How the Hugging Face Speech-to-Speech Pipeline Handles Concurrent WebSocket Connections in Realtime Mode](/huggingface/speech-to-speech/realtime-mode-concurrent-websocket-connections-pooling)

Learn how the Hugging Face Speech-to-Speech pipeline manages concurrent WebSocket connections with a fixed pool of PipelineUnits for thread-safe, real-time audio processing.

- Tags: internals
- Published: 2026-07-30

### [Hugging Face Speech-to-Speech Run Modes: Local, Socket, WebSocket, and Realtime Explained](/huggingface/speech-to-speech/speech-to-speech-run-modes-realtime-local-websocket-socket-comparison)

Explore Hugging Face speech-to-speech run modes local, socket, websocket, and realtime. Understand their differences and choose the best mode for your audio streaming needs.

- Tags: deep-dive
- Published: 2026-07-30

### [How the VAD → STT → LLM → TTS Pipeline Uses Typed Queues for Inter-Component Communication](/huggingface/speech-to-speech/vad-stt-llm-tts-pipeline-inter-component-communication-queues)

Discover how the VAD STT LLM TTS pipeline uses typed queues for efficient inter-component communication. Learn about Pydantic messages and decoupled handler components.

- Tags: architecture
- Published: 2026-07-30

### [How to Implement Response Streaming with Responses API Backend in Speech-to-Speech](/huggingface/speech-to-speech/how-to-implement-response-streaming-responses-api-backend)

Implement response streaming with the Responses API backend. Enable real-time token delivery in speech-to-speech for incremental TextDelta events. Learn how to set responses_api_stream=True.

- Tags: how-to-guide
- Published: 2026-07-30

### [How to Implement Bilingual Voice Conversations with Language Prompts](/huggingface/speech-to-speech/how-to-implement-bilingual-voice-conversations-language-prompts)

Implement bilingual voice conversations using language prompts. Pass language codes via TTSInput to enable automatic language switching in real-time speech applications.

- Tags: how-to-guide
- Published: 2026-07-30

### [How to Implement Custom Audio Streaming for Embedded Devices with Speech-to-Speech](/huggingface/speech-to-speech/how-to-implement-custom-audio-streaming-embedded-devices)

Implement custom audio streaming for embedded devices using huggingface/speech-to-speech. Stream 16 kHz PCM audio via raw TCP sockets without complex protocols.

- Tags: how-to-guide
- Published: 2026-07-30

### [How to Configure HF Inference Providers as LLM Backends in the Speech-to-Speech Library](/huggingface/speech-to-speech/how-to-configure-hf-inference-providers-llm-backends)

Configure HF Inference Providers as LLM backends in speech-to-speech. Learn to set flags like --llm-provider hf, --hf-endpoint, and --hf-token for seamless integration.

- Tags: how-to-guide
- Published: 2026-07-30

### [How the Speech-to-Speech Pipeline Handles Soft-Ended Turns and Turn Reopening](/huggingface/speech-to-speech/how-does-pipeline-handle-soft-ended-turns-turn-reopening)

Discover how the speech-to-speech pipeline manages soft-ended turns and turn reopening using VADHandler and SpeculativeTurnTracker for a seamless conversational experience.

- Tags: internals
- Published: 2026-07-30

### [How to Run the Speech-to-Speech Pipeline in Docker with GPU Support](/huggingface/speech-to-speech/how-to-run-speech-to-speech-pipeline-docker-gpu)

Easily run the speech-to-speech pipeline in Docker with GPU acceleration. Deploy the Hugging Face model quickly using Docker Compose and NVIDIA Container Toolkit for optimal performance.

- Tags: how-to-guide
- Published: 2026-07-30

### [How to Implement Custom Chat Prompts and Conversation Memory in Speech-to-Speech](/huggingface/speech-to-speech/how-to-implement-custom-chat-prompts-conversation-memory)

Learn to implement custom chat prompts and conversation memory in speech-to-speech with Hugging Face. Customize system prompts and manage chat history efficiently for better AI interactions.

- Tags: how-to-guide
- Published: 2026-07-30

### [How the cancel_scope Mechanism Interrupts In-Progress Responses in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-does-cancel-scope-mechanism-interrupt-in-progress-responses)

Learn how the cancel scope mechanism in Hugging Face Speech-to-Speech instantly aborts in-progress responses by updating a counter and discard flag without blocking producer threads.

- Tags: internals
- Published: 2026-07-30

### [How to Implement VAD with DeepFilterNet Audio Enhancement in Speech-to-Speech](/huggingface/speech-to-speech/how-to-implement-vad-deepfilternet-audio-enhancement)

Learn to implement VAD with DeepFilterNet audio enhancement in Speech-to-Speech. Discover how the VADHandler simplifies voice activity detection and noise reduction for cleaner audio.

- Tags: how-to-guide
- Published: 2026-07-30

### [How to Configure CUDA Settings for Qwen3-TTS on Linux](/huggingface/speech-to-speech/how-to-configure-cuda-settings-qwen3-tts-linux)

Optimize Qwen3-TTS on Linux by configuring CUDA settings. Set device to cuda, backend to torch and dtype to float16 or auto for efficient GPU acceleration.

- Tags: how-to-guide
- Published: 2026-07-30

### [How Speculative Turn Tracking Improves Conversation Flow in Speech-to-Speech Systems](/huggingface/speech-to-speech/how-does-speculative-turn-tracking-improve-conversation-flow)

Speculative turn tracking enhances speech-to-speech systems by monitoring soft-ended pauses, preventing interruptions and improving conversation flow. Learn how it works.

- Tags: deep-dive
- Published: 2026-07-30

### [How to Swap STT Backends in the Hugging Face Speech-to-Speech Library](/huggingface/speech-to-speech/how-to-swap-stt-backends-parakeet-whisper-faster-whisper-paraformer)

Easily swap STT backends like Whisper FasterWhisper and Parakeet in the Hugging Face Speech-to-Speech library Select your preferred engine with ModuleArguments for flexible ASR.

- Tags: how-to-guide
- Published: 2026-07-30

### [How to Debug Latency Issues in the VAD-STT-LLM-TTS Pipeline: A Step-by-Step Guide](/huggingface/speech-to-speech/how-to-debug-latency-issues-vad-stt-llm-tts-pipeline)

Debug latency issues in VAD-STT-LLM-TTS pipelines with perf_counter timing and debug logging. Isolate delays by disabling optional features to optimize speech processing.

- Tags: how-to-guide
- Published: 2026-07-30

### [How to Implement Tool Calling and Function Calling with the Language Model](/huggingface/speech-to-speech/how-to-implement-tool-calling-function-calling-language-model)

Learn to implement tool calling and function calling with language models. Use the huggingface speech-to-speech library to execute Python functions and get OpenAI-compatible tool call objects.

- Tags: how-to-guide
- Published: 2026-07-30

### [How Live Transcription Works During Active Speech Detection in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-does-live-transcription-work-active-speech-detection)

Discover how live transcription leverages active speech detection in Hugging Face Speech-to-Speech. Learn how VAD buffers audio and processes segments for accurate real-time transcription.

- Tags: internals
- Published: 2026-07-30

### [How to Implement Custom TTS Voice Cloning with Pocket TTS in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-implement-custom-tts-voice-cloning-pocket-tts)

Learn to implement custom TTS voice cloning with Pocket TTS using Hugging Face speech-to-speech. Easily set voice embeddings from presets, local files, or repo URLs.

- Tags: how-to-guide
- Published: 2026-07-30

### [How to Optimize Speech-to-Speech for Apple Silicon with MLX](/huggingface/speech-to-speech/how-to-optimize-speech-to-speech-apple-silicon-mlx)

Optimize speech-to-speech on Apple Silicon using MLX. Configure the backend for LLM STT and TTS components to route inference through Metal Performance Shaders and serialize concurrent operations.

- Tags: performance
- Published: 2026-07-30

### [How Pipeline Pooling Handles Multiple Concurrent WebSocket Connections in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-does-pipeline-pooling-handle-concurrent-websocket-connections)

Discover how pipeline pooling manages concurrent WebSocket connections in Hugging Face Speech-to-Speech. Learn about isolated PipelineUnit instances and strict concurrency control.

- Tags: internals
- Published: 2026-07-30

### [How to Implement Multi-Language Voice Conversations with Auto-Switching in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-to-implement-multi-language-voice-conversations-auto-switching)

Implement multi-language voice conversations with auto-switching using Hugging Face. Detect languages with lingua py and dynamically swap TTS models in real-time.

- Tags: how-to-guide
- Published: 2026-07-30

### [How `--chatsize` Controls Conversation Memory in the Speech-to-Speech LLM](/huggingface/speech-to-speech/how-chat-size-affect-conversation-memory-llm-speech-to-speech)

Discover how --chatsize controls conversation memory in the speech-to-speech LLM. Learn to manage dialogue history for better model performance and understand its impact on generation.

- Tags: deep-dive
- Published: 2026-07-11

### [What Is the `--short_segment_merge_ms` VAD Parameter in Speech-to-Speech?](/huggingface/speech-to-speech/what-is-short-segment-merge-ms-vad-parameter-speech-to-speech)

Discover the `--short_segment_merge_ms` VAD parameter in speech-to-speech. Learn how it merges brief VAD segments to prevent speech splitting during short pauses.

- Tags: deep-dive
- Published: 2026-07-11

### [How to Use Custom Voice Cloning with Pocket TTS in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-use-custom-voice-cloning-pocket-tts-huggingface-speech-to-speech)

Learn how to use custom voice cloning with Pocket TTS in Hugging Face Speech-to-Speech. Clone voices from local files or Hugging Face repos effortlessly for your projects.

- Tags: how-to-guide
- Published: 2026-07-11

### [What Is the TranscriptionNotifier and How Does It Manage Text Output Queues in Speech-to-Speech?](/huggingface/speech-to-speech/what-is-transcription-notifier-how-it-manages-text-output-queues-speech-to-speech)

Discover the TranscriptionNotifier's role in speech-to-speech systems. Learn how it manages text output queues, bridges STT and LLM, and emits transcription events.

- Tags: internals
- Published: 2026-07-11

### [How the Speech-to-Speech Pipeline Handles Graceful Shutdown with Signal Handlers](/huggingface/speech-to-speech/how-speech-to-speech-pipeline-handle-graceful-shutdown-signal-handlers)

Learn how the Hugging Face speech-to-speech pipeline achieves graceful shutdown using SIGINT and SIGTERM signal handlers. Ensure clean thread exits with a 5-second timeout.

- Tags: internals
- Published: 2026-07-11

### [What Is RuntimeConfig in Speech-to-Speech? Configuring Session Parameters](/huggingface/speech-to-speech/what-is-runtimeconfig-how-it-configures-session-parameters-speech-to-speech)

Discover RuntimeConfig in speech-to-speech. Learn how this mutable object manages Realtime session state and conversation history for seamless WebSocket connections.

- Tags: internals
- Published: 2026-07-11

### [How the VAD Threshold (--thresh) Controls Turn-Taking in Speech-to-Speech](/huggingface/speech-to-speech/how-vad-threshold-thresh-interact-turn-taking-behavior-speech-to-speech)

Learn how the VAD threshold --thresh controls turn-taking in speech-to-speech conversations. Adjust this setting to manage when user turns begin and how turn reopening behaves.

- Tags: deep-dive
- Published: 2026-07-11

### [What Is the ThreadManager and How It Orchestrates Handler Threads in Speech-to-Speech](/huggingface/speech-to-speech/what-is-threadmanager-how-it-orchestrates-handler-threads-speech-to-speech)

Discover how the ThreadManager orchestrates handler threads in Hugging Face speech-to-speech, managing worker thread lifecycles for efficient startup, monitoring, and shutdown.

- Tags: internals
- Published: 2026-07-11

### [How to Configure vLLM with Tool Calling for the Chat-Completions Backend in Speech-to-Speech](/huggingface/speech-to-speech/how-configure-vllm-tool-calling-chat-completions-backend-speech-to-speech)

Configure vLLM with tool calling for the chat-completions backend in speech-to-speech. Use ChatCompletionsApiModelHandler with environment variables and the chat-completions flag.

- Tags: how-to-guide
- Published: 2026-07-11

### [WebSocket vs Socket Modes in HuggingFace Speech-to-Speech: Key Differences](/huggingface/speech-to-speech/what-is-difference-between-websocket-and-socket-modes-huggingface-speech-to-speech)

Understand WebSocket vs Socket modes in HuggingFace Speech-to-Speech. WebSocket offers structured events for OpenAI API, while Socket provides minimal TCP for raw audio.

- Tags: deep-dive
- Published: 2026-07-11

### [How Does --enable_lang_prompt Work with Language Auto-Detection in Speech-to-Speech?](/huggingface/speech-to-speech/how-enable-lang-prompt-work-language-auto-detection-speech-to-speech)

Learn how the enable_lang_prompt flag in speech-to-speech works with language auto-detection. Ensure your model responds in the correct language with this guide.

- Tags: deep-dive
- Published: 2026-07-11

### [Speech-to-Speech Queue Types: How AudioInItem, TTSInItem, and Typed Queues Connect Pipeline Handlers](/huggingface/speech-to-speech/what-are-queue-types-audioinitem-tts-initem-how-they-connect-handlers-speech-to-speech)

Understand speech-to-speech queue types like AudioInItem and TTSInItem. Learn how these typed queues ensure type-safe data flow between pipeline handlers in the Hugging Face library.

- Tags: internals
- Published: 2026-07-11

### [PipelineUnit Architecture for Realtime Mode in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/what-is-pipeline-unit-architecture-realtime-mode-huggingface-speech-to-speech)

Explore the PipelineUnit architecture for real-time speech-to-speech in Hugging Face. Learn how isolated units enable concurrent, state-separated conversations for a seamless user experience.

- Tags: architecture
- Published: 2026-07-11

### [How to Set Up a Fully Local Speech-to-Speech Stack with llama.cpp Instead of OpenAI API](/huggingface/speech-to-speech/how-set-up-fully-local-stack-llama-cpp-instead-openai-api-speech-to-speech)

Create a local speech-to-speech stack with llama.cpp by replacing the OpenAI API. Learn how to set up your offline voice conversations easily and efficiently.

- Tags: how-to-guide
- Published: 2026-07-11

### [How `--qwen3_tts_backend` GGML vs Torch Affects Speech-to-Speech Performance](/huggingface/speech-to-speech/how-qwen3-tts-backend-ggml-vs-torch-options-affect-performance-speech-to-speech)

Explore how --qwen3_tts_backend GGML vs Torch impacts speech-to-speech performance. Discover Torch's lower latency benefits and GGML's wider compatibility for your projects.

- Tags: performance
- Published: 2026-07-11

### [LMOutputProcessor: How It Transforms LLM Output for TTS in Speech-to-Speech](/huggingface/speech-to-speech/what-is-lm-output-processor-how-it-transforms-llm-output-tts-speech-to-speech)

Learn how LMOutputProcessor transforms LLM output for TTS in speech-to-speech. This essential component bridges LLMs and TTS, ensuring seamless conversion for real-time speech.

- Tags: internals
- Published: 2026-07-11

### [MLX Contention on Apple Silicon in Speech-to-Speech: How --num_pipelines Interacts with It](/huggingface/speech-to-speech/what-causes-mlx-contention-apple-silicon-num-pipelines-interaction-speech-to-speech)

Learn how MLX contention on Apple Silicon impacts speech-to-speech inference and how --num_pipelines prevents crashes by enforcing single-pipeline mode.

- Tags: performance
- Published: 2026-07-11

### [How `live_transcription_update_interval` Controls Streaming Transcription Latency in Speech-to-Speech](/huggingface/speech-to-speech/how-does-live-transcription-update-interval-affect-streaming-transcription-latency-speech-to-speech)

Learn how live_transcription_update_interval controls streaming transcription latency in speech-to-speech. Adjust this setting for faster, more responsive audio processing.

- Tags: performance
- Published: 2026-07-11

### [Responses API vs Chat Completions LLM Backends in HuggingFace Speech-to-Speech](/huggingface/speech-to-speech/what-is-difference-between-responses-api-and-chat-completions-llm-backends-in-huggingface-speech-to-speech)

Compare Responses API vs Chat Completions LLM backends in HuggingFace Speech-to-Speech. Understand differences in tool schemas and streaming for efficient integration.

- Tags: comparisons
- Published: 2026-07-11

### [How CancelScope Enables Interruption of LLM and TTS Responses in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-does-cancel-scope-enable-interruption-of-llm-and-tts-responses-in-speech-to-speech-pipeline)

Discover how CancelScope instantly invalidates LLM text and TTS audio with a generation counter and discarding flag, enabling seamless interruption in speech-to-speech pipelines.

- Tags: internals
- Published: 2026-07-11

### [What Is the SpeculativeTurnTracker and How Does It Manage Uncommitted Turns in Speech‑to‑Speech?](/huggingface/speech-to-speech/what-is-speculative-turn-tracker-and-how-does-it-manage-uncommitted-turns-in-speech-to-speech)

Discover the SpeculativeTurnTracker from Hugging Face Speech-to-Speech. Learn how this state machine manages uncommitted turns and prevents race conditions for seamless conversations.

- Tags: internals
- Published: 2026-07-11

### [How the Hugging Face Speech-to-Speech Pipeline Handles Concurrent WebSocket Connections with num_pipelines](/huggingface/speech-to-speech/how-does-huggingface-speech-to-speech-pipeline-handle-concurrent-websocket-connections-with-num-pipelines)

Learn how the Hugging Face speech-to-speech pipeline efficiently manages concurrent WebSocket connections using num_pipelines and thread pools for optimal performance.

- Tags: internals
- Published: 2026-07-11

### [Optimizing TTS Streaming Latency in the Hugging Face Speech-to-Speech Library](/huggingface/speech-to-speech/best-practices-optimizing-tts-streaming-latency)

Optimize TTS streaming latency with small chunk sizes, streaming mode, minimal audio buffers, and speculative turns to achieve sub-100ms first-audio latency on GPUs. Learn best practices.

- Tags: best-practices
- Published: 2026-07-10

### [How to Implement Custom Tool Definitions with the Local Transformers/MLX-LM LLM Backend](/huggingface/speech-to-speech/how-to-implement-custom-tool-definitions-local-llm)

Learn to implement custom tool definitions with the local transformers/mlx-lm LLM backend. Wrap Python functions in FunctionTool and enable local LLMs to call external capabilities.

- Tags: tutorial
- Published: 2026-07-10

### [Understanding TranscriptionNotifier: How It Enables Live Transcription Events in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/purpose-of-transcriptionnotifier-and-live-transcription-events)

Discover TranscriptionNotifier and how it powers live speech-to-text transcription events in Hugging Face. Learn about its real-time event emission and execution modes.

- Tags: how-to-guide
- Published: 2026-07-10

### [How the Speech-to-Speech Pipeline Handles Audio Format Conversion from Client PCM to 16kHz](/huggingface/speech-to-speech/audio-format-conversion-pcm-to-16khz)

Discover how the huggingface speech-to-speech pipeline seamlessly converts client PCM audio to 16kHz using TorchAudio Resample transform before processing and after transmission.

- Tags: internals
- Published: 2026-07-10

### [Language Coverage Differences Between STT Backends in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/language-coverage-differences-stt-backends)

Explore STT backend language coverage in Hugging Face speech-to-speech. Compare Whisper 12 languages, Parakeet TDT 25 European languages, and Paraformer for Mandarin.

- Tags: comparison
- Published: 2026-07-10

### [How Silero VAD Determines Speech Boundaries and Emits speech_started / speech_stopped Events](/huggingface/speech-to-speech/how-silero-vad-determines-speech-boundaries-and-emits-events)

Learn how Silero VAD detects speech boundaries by comparing probabilities to a threshold and emits speech_started and speech_stopped events through its VADHandler.

- Tags: how-to-guide
- Published: 2026-07-10

### [RuntimeConfig Deep-Merge Explained: How session.update Events Configure Speech-to-Speech Sessions](/huggingface/speech-to-speech/purpose-of-runtimeconfig-and-configuration-merging)

Discover how RuntimeConfig in huggingface/speech-to-speech uses deep-merge with session.update events to efficiently configure speech-to-speech sessions without overwriting existing settings.

- Tags: deep-dive
- Published: 2026-07-10

### [How to Configure Custom Voice Cloning with Pocket TTS in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/how-to-configure-custom-voice-cloning-pocket-tts)

Learn how to configure custom voice cloning with Pocket TTS in the speech-to-speech pipeline. Easily set preset names, local files, or repo URLs for unique voice generation.

- Tags: how-to-guide
- Published: 2026-07-10

### [Deploying Speech-to-Speech in Production: Key Considerations for the Hugging Face Pipeline](/huggingface/speech-to-speech/production-deployment-considerations-speech-to-speech)

Learn key considerations for speech-to-speech production deployment. Optimize hardware, manage latency with CancelScope, and choose the right deployment mode for your Hugging Face pipeline.

- Tags: best-practices
- Published: 2026-07-10

### [How the Speech-to-Speech Pipeline Handles Streaming Audio Memory Management](/huggingface/speech-to-speech/audio-memory-management-streaming-audio)

Learn how Hugging Face's speech-to-speech pipeline manages streaming audio memory to prevent unbounded queue growth using back-pressure, sentinel values, and conditional queues.

- Tags: internals
- Published: 2026-07-10

### [What Is the ThreadManager in Hugging Face Speech-to-Speech?](/huggingface/speech-to-speech/what-is-threadmanager-and-handler-thread-coordination)

Learn about Hugging Face's ThreadManager, an orchestration utility that expertly manages handler threads in the Speech-to-Speech pipeline, ensuring smooth startup and graceful shutdown.

- Tags: internals
- Published: 2026-07-10

### [Latency and Voice Quality Tradeoffs Between TTS Backends in Speech-to-Speech](/huggingface/speech-to-speech/latency-voice-quality-tradeoffs-tts-backends)

Explore latency vs voice quality tradeoffs between 5 TTS backends in Hugging Face Speech-to-Speech. Discover options from robotic to high-fidelity synthesis with MLX acceleration.

- Tags: performance
- Published: 2026-07-10

### [How the Speech-to-Speech Pipeline Handles Device Allocation Across VAD, STT, LLM, and TTS: CUDA, MPS, and CPU Support](/huggingface/speech-to-speech/pipeline-device-allocation-cuda-mps-cpu)

Discover how the speech-to-speech pipeline manages device allocation for VAD, STT, LLM, and TTS on CUDA, MPS, and CPU. Learn about macOS optimizations and automatic CPU fallback.

- Tags: internals
- Published: 2026-07-10

### [Performance and Memory Implications of Using `--num_pipelines` with Multiple Concurrent WebSocket Connections](/huggingface/speech-to-speech/performance-memory-implications-of-num-pipelines)

Explore performance and memory implications of Hugging Face speech-to-speech `--num_pipelines`. Understand how increasing pipelines impacts latency, memory, and GPU/CPU usage with concurrent WebSocket connections.

- Tags: performance
- Published: 2026-07-10

### [OpenAI Realtime Event System: Tool Calling Differences Between Local LLM and API Backends](/huggingface/speech-to-speech/openai-realtime-event-system-tool-calling-differences)

Explore OpenAI tool calling differences: API backends use discrete events, local LLMs stream deltas. Learn how the realtime event system manages state.

- Tags: how-to-guide
- Published: 2026-07-10

### [LMOutputProcessor in Hugging Face Speech-to-Speech: Splitting Text and Tool Calls for TTS](/huggingface/speech-to-speech/what-is-lmoutputprocessor-and-text-splitting)

Learn how Hugging Face Speech-to-Speech's LMOutputProcessor splits LLM output, separating tool calls from text before sending clean speech data to TTS. Optimize your TTS pipeline.

- Tags: deep-dive
- Published: 2026-07-10

### [How to Add a Custom STT Backend to the Speech-to-Speech Pipeline: A Complete Guide](/huggingface/speech-to-speech/how-to-add-custom-stt-backend)

Learn how to add a custom STT backend to the huggingface speech-to-speech pipeline. Implement a handler class and register it for seamless integration. Complete guide.

- Tags: how-to-guide
- Published: 2026-07-10

### [Speech-to-Speech responses-api vs chat-completions LLM backends: Key Differences and When to Use Each](/huggingface/speech-to-speech/differences-between-responses-api-and-chat-completions-llm-backends)

Explore responses-api vs chat-completions LLM backends. Understand key differences in message conversion, reasoning, and streaming to choose the right Speech-to-Speech solution.

- Tags: deep-dive
- Published: 2026-07-10

### [How the `--language auto` Flag Works in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/how-language-auto-flag-works-and-language-detection)

Learn how the --language auto flag in Hugging Face Speech-to-Speech enables automatic language detection. Discover how STT processes and forwards language codes to the LLM for seamless audio translation.

- Tags: how-to-guide
- Published: 2026-07-10

### [SpeculativeTurnTracker: How Hugging Face Reduces Latency in Speech-to-Speech Pipelines](/huggingface/speech-to-speech/what-is-speculativeturntracker-and-how-it-reduces-latency)

Discover how SpeculativeTurnTracker by Hugging Face slashes speech-to-speech latency. This revision manager enables speculative turns for faster audio responses, correcting errors instantly.

- Tags: internals
- Published: 2026-07-10

### [How to Configure Custom VAD Thresholds for Specific Acoustic Environments in Speech-to-Speech](/huggingface/speech-to-speech/how-to-configure-custom-vad-thresholds)

Learn to configure custom VAD thresholds for speech-to-speech in specific acoustic environments. Adjust thresholds statically or dynamically with ease. Optimize your speech processing now.

- Tags: how-to-guide
- Published: 2026-07-10

### [CancelScope Mechanism: Managing Response Interruption Across Speech-to-Speech Pipeline Threads](/huggingface/speech-to-speech/what-is-cancelscope-mechanism-and-how-it-manages-interruption)

Learn about the CancelScope mechanism in huggingface speech-to-speech. This primitive manages response interruption across pipeline threads, stopping stale LLM and TTS outputs without locks.

- Tags: internals
- Published: 2026-07-10

### [Handling Concurrent WebSocket Connections in Realtime Mode with PipelineUnit Pools](/huggingface/speech-to-speech/how-pipeline-architecture-handle-concurrent-websocket-connections)

Discover how huggingface speech-to-speech handles concurrent WebSocket connections using isolated PipelineUnit pools for efficient realtime processing and full session isolation.

- Tags: internals
- Published: 2026-07-10

### [Configuring MPS Device and MLX Backends with --local_mac_optimal_settings in Speech-to-Speech](/huggingface/speech-to-speech/configure-mps-mlx-backends-local-mac-optimal-settings)

Optimize your Speech-to-Speech pipeline on Apple Silicon Macs. Discover how `--local_mac_optimal_settings` configures MPS device and MLX backends for faster STT, LLM, and TTS.

- Tags: how-to-guide
- Published: 2026-07-09

### [How Pool Size (`--num_pipelines`) Affects Concurrent Session Capacity in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/pipeline-pool-size-concurrent-sessions)

Discover how increasing --num_pipelines boosts concurrent session capacity in Hugging Face Speech-to-Speech. Learn its limitations for realtime sessions and Apple Silicon.

- Tags: performance
- Published: 2026-07-09

### [How to Debug Latency Issues in the Speech-to-Speech Pipeline: A Complete Guide](/huggingface/speech-to-speech/debug-latency-issues-speech-to-speech-pipeline)

Debug speech-to-speech pipeline latency by monitoring queues optimizing the VAD STT LLM TTS handler chain and identifying MLX lock contention on macOS Learn how to speed up your ASR.

- Tags: how-to-guide
- Published: 2026-07-09

### [How to Handle and Stream Tool Calls from LLM in a Speech-to-Speech Pipeline](/huggingface/speech-to-speech/handle-stream-tool-calls-llm-pipeline)

Master real time tool calls in speech to speech pipelines. Learn to inject tool definitions, detect code blocks, and parse arguments for seamless LLM integration.

- Tags: how-to-guide
- Published: 2026-07-09

### [How to Configure Multi-Language Support with Automatic Language Detection in Speech-to-Speech](/huggingface/speech-to-speech/multi-language-support-automatic-language-detection)

Configure multi language support for speech to speech using automatic language detection with lingua py and Parakeet TDT. Automatically detect 25 European languages.

- Tags: how-to-guide
- Published: 2026-07-09

### [Architecture of the Speculative Turns System for Uncommitted Speech in Speech-to-Speech](/huggingface/speech-to-speech/speculative-turns-system-architecture-uncommitted-speech)

Explore the architecture of Hugging Face's speculative turns system for uncommitted speech. Learn how it efficiently manages turn revisions and late-arriving audio for seamless context.

- Tags: architecture
- Published: 2026-07-09

### [Live Transcription with Parakeet TDT Controls and Parameters: A Developer’s Guide to Real-Time Speech-to-Text](/huggingface/speech-to-speech/live-transcription-parakeet-tdt-controls)

Master live transcription with Parakeet TDT controls. This guide details Hugging Face pipeline parameters like enable_live_transcription for real-time speech-to-text.

- Tags: how-to-guide
- Published: 2026-07-09

### [Quantization Options for Qwen3-TTS with MLX: Impact on Latency and Audio Quality](/huggingface/speech-to-speech/qwen3-tts-mlx-quantization-latency-quality)

Explore Qwen3-TTS MLX quantization options bf16 4bit 6bit 8bit. Reduce model size and latency by up to 30% while maintaining audio quality.

- Tags: performance
- Published: 2026-07-09

### [Implementing Interrupt Handling for Turn-Taking in Speech-to-Speech: A Complete Technical Guide](/huggingface/speech-to-speech/interrupt-handling-turn-taking-speech-to-speech)

Learn to implement interrupt handling for turn-taking in speech-to-speech with Hugging Face. This guide details barge-in functionality and response management for real-time applications.

- Tags: how-to-guide
- Published: 2026-07-09

### [Performance Implications of Running STT, LLM, and TTS on Same vs Separate Devices](/huggingface/speech-to-speech/performance-stt-llm-tts-same-vs-separate-devices)

Compare performance of running STT LLM TTS on same vs separate devices. Discover how device allocation impacts GPU memory, PCIe transfers, and overall latency for your speech applications.

- Tags: performance
- Published: 2026-07-09

### [OpenAI Realtime Protocol Handler: Managing Concurrent WebSocket Sessions in Speech-to-Speech](/huggingface/speech-to-speech/openai-realtime-protocol-handler-websocket-sessions)

Learn how the Hugging Face Speech-to-Speech library manages concurrent WebSocket sessions using asyncio Semaphore and RuntimeConfig for efficient real-time communication. Optimize your performance today.

- Tags: architecture
- Published: 2026-07-09

### [Difference Between responses-api and chat-completions LLM Backends in Speech-to-Speech](/huggingface/speech-to-speech/responses-api-vs-chat-completions-llm-backends)

Understand the difference between responses-api and chat-completions LLM backends in speech-to-speech on Hugging Face. Learn about legacy vs modern API features and tool call streaming.

- Tags: deep-dive
- Published: 2026-07-09

### [Configuring Silero VAD Parameters for Different Acoustic Environments in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/configure-silero-vad-parameters-acoustic-environments)

Learn to configure Silero VAD parameters like threshold and min_silence_duration_ms for diverse acoustic environments within Hugging Face speech-to-speech. Adapt in real-time.

- Tags: how-to-guide
- Published: 2026-07-09

### [How to Benchmark and Profile the Different Pipeline Components for Optimization in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/benchmark-profile-pipeline-components-optimization)

Learn to benchmark and profile Hugging Face Speech-to-Speech pipeline components for optimization. Use provided scripts and profilers to measure latency, throughput, and RTF for peak performance.

- Tags: performance
- Published: 2026-07-08

### [Threading Model and Queue-Based Communication in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/threading-model-handler-threads-queues)

Understand Hugging Face Speech-to-Speech threading. Learn how handler threads use thread-safe queues for efficient audio chunk and message communication between processing stages.

- Tags: internals
- Published: 2026-07-08

### [Security Considerations When Exposing the Speech-to-Speech WebSocket Server](/huggingface/speech-to-speech/security-considerations-websocket-server)

Secure your Speech-to-Speech WebSocket server with authentication, TLS, input validation, and connection limits. Prevent unauthorized access and protect your data.

- Tags: security-best-practices
- Published: 2026-07-08

### [How to Configure Live Transcription and Transcription Events in the Realtime API](/huggingface/speech-to-speech/configure-live-transcription-realtime-api-events)

Learn to configure live transcription and events in the Realtime API. Enable live transcription, set intervals, and emit partial and completed events for seamless audio processing with huggingface/speech-to-speech.

- Tags: how-to-guide
- Published: 2026-07-08

### [How Speculative Turns Handle Multiple In-Flight Responses in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/speculative-turns-in-flight-responses-handling)

Discover how Hugging Face's speculative turns manage multiple in-flight responses with a thread-safe tracker, ensuring reliable speech-to-speech processing and preventing race conditions.

- Tags: internals
- Published: 2026-07-08

### [How to Integrate a Custom TTS or STT Handler into the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/integrate-custom-tts-stt-handler)

Integrate custom TTS or STT handlers into the speech-to-speech pipeline by subclassing, implementing methods, and registering handlers for enhanced voice AI applications.

- Tags: how-to-guide
- Published: 2026-07-08

### [How to Configure VAD Parameters in Hugging Face Speech-to-Speech: Thresholds, Silence Detection, and Realtime Settings](/huggingface/speech-to-speech/configure-vad-parameters)

Learn to configure VAD parameters in Hugging Face Speech-to-Speech. Master thresholds, silence detection, and realtime settings for optimal performance. Get started today.

- Tags: how-to-guide
- Published: 2026-07-08

### [Session Lifecycle and PipelineUnit Pool Management in Speech-to-Speech Realtime API](/huggingface/speech-to-speech/session-lifecycle-pipelineunit-pool)

Understand the session lifecycle in the Speech-to-Speech realtime API. Learn how the PipelineUnit pool manages concurrent connections with claim, drain, and release to ensure isolation.

- Tags: internals
- Published: 2026-07-08

### [How CancelScope Manages Concurrent Responses and Handles Race Conditions in Speech-to-Speech](/huggingface/speech-to-speech/cancelscope-concurrent-responses-race-conditions)

Discover how CancelScope prevents race conditions in speech-to-speech by managing concurrent responses with a generation counter and atomic flags.

- Tags: how-to-guide
- Published: 2026-07-08

### [How to Run Speech-to-Speech on Apple Silicon (MPS/MLX), CUDA, or CPU: A Complete Hardware Guide](/huggingface/speech-to-speech/run-speech-to-speech-apple-silicon-cuda-cpu)

Discover how to run speech-to-speech on Apple Silicon MPS MLX CUDA or CPU by setting the device flag. Optimize your hardware for STT LLM and TTS components. Get the complete guide.

- Tags: how-to-guide
- Published: 2026-07-08

### [How to Set Up Multi-Language Support with Automatic Language Detection in Speech-to-Speech](/huggingface/speech-to-speech/multi-language-support-automatic-detection)

Enable automatic language detection for speech-to-speech translation across 25 European languages. Install lingua-py, use Parakeet TDT handler, and set language to auto.

- Tags: how-to-guide
- Published: 2026-07-08

### [How Interruption Handling and Barge-In Work in the Hugging Face Speech-to-Speech Pipeline](/huggingface/speech-to-speech/interruption-handling-barge-in)

Understand interruption handling and barge-in in the Hugging Face speech-to-speech pipeline. See how user speech cancels assistant output for a seamless experience.

- Tags: internals
- Published: 2026-07-08

### [How the VAD → STT → LLM → TTS Pipeline Works in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/vad-stt-llm-tts-pipeline-architecture)

Understand the VAD STT LLM TTS pipeline in Hugging Face Speech-to-Speech. Explore this asynchronous, real-time architecture for low-latency voice processing.

- Tags: architecture
- Published: 2026-07-08

### [Real-Time Speech-to-Speech Conversion: A Complete Implementation Guide](/huggingface/speech-to-speech/real-time-speech-to-speech-conversion-examples)

Implement real-time speech-to-speech conversion with the complete pipeline from huggingface. This guide shows live audio transcription, response generation, and speech synthesis for ongoing conversations.

- Tags: how-to-guide
- Published: 2026-07-07

### [How to Improve Speech-to-Speech Inference Latency: A Technical Guide for the Hugging Face Pipeline](/huggingface/speech-to-speech/improve-latency-speech-to-speech-inference)

Learn how to reduce speech-to-speech inference latency with efficient device selection speculative turns and queue tuning in the Hugging Face pipeline Optimize your system now

- Tags: performance
- Published: 2026-07-07

### [Best Practices for the Hugging Face Speech-to-Speech Library](/huggingface/speech-to-speech/best-practices-for-using-speech-to-speech-library)

Master the Hugging Face Speech-to-Speech library. Learn to optimize deployment, reduce latency with efficient back-ends, and enhance your real-time applications. Get started now.

- Tags: best-practices
- Published: 2026-07-07

### [How to Customize Speaker Identity in Speech-to-Speech: CLI and API Methods](/huggingface/speech-to-speech/customize-speaker-identity-speech-to-speech)

Customize speaker identity in speech-to-speech with CLI and API. Learn to set speaker arguments or session voice fields for personalized voice generation.

- Tags: how-to-guide
- Published: 2026-07-07

### [Speech-to-Speech Model Architecture: Inside the Hugging Face Real-Time Voice Pipeline](/huggingface/speech-to-speech/architecture-of-speech-to-speech-models)

Explore the four-stage speech-to-speech model architecture from Hugging Face: VAD, STT, LLM, and TTS. Enable real-time voice conversations with this low-latency pipeline.

- Tags: architecture
- Published: 2026-07-07

### [Where to Find Pre-Trained Speech-to-Speech Models: Hugging Face Hub Integration Guide](/huggingface/speech-to-speech/where-to-find-pretrained-speech-to-speech-models)

Discover pre-trained speech-to-speech models on Hugging Face Hub. Easily integrate VAD, STT, LLM, and TTS components using the CLI or Python API. Download weights automatically.

- Tags: getting-started
- Published: 2026-07-07

### [How to Handle Noisy Input Audio in the Speech-to-Speech Pipeline](/huggingface/speech-to-speech/handle-noisy-input-audio-speech-to-speech)

Handle noisy input audio in speech-to-speech pipelines with noise-floor thresholds, buffering, and DeepFilterNet. Learn how to configure VADHandlerArguments for clear audio.

- Tags: how-to-guide
- Published: 2026-07-07

### [Speech Transformations in the Hugging Face Speech-to-Speech Pipeline](/huggingface/speech-to-speech/supported-speech-transformations-speech-to-speech)

Explore Hugging Face speech transformations including voice cloning, custom speakers, voice design, and multilingual TTS. Transform audio streams with new characteristics, languages, and styles.

- Tags: deep-dive
- Published: 2026-07-07

### [How to Evaluate the Performance of a Speech-to-Speech Model: Benchmarking Guide](/huggingface/speech-to-speech/evaluate-performance-of-speech-to-speech-model)

Learn how to evaluate speech-to-speech model performance with our benchmarking guide. Measure warm-up time, latency, and real-time factor using huggingface/speech-to-speech scripts.

- Tags: benchmarking-guide
- Published: 2026-07-07

### [System Requirements for Running Speech-to-Speech Models: Hardware, Software, and Setup Guide](/huggingface/speech-to-speech/system-requirements-for-running-speech-to-speech-models)

Discover system requirements for Hugging Face speech-to-speech models. Learn about Python 3.10+, GPU/MLX needs, and OS compatibility for NLP projects. Get started now.

- Tags: getting-started
- Published: 2026-07-07

### [How to Integrate Speech-to-Speech into a Python Application: A Complete Guide](/huggingface/speech-to-speech/integrate-speech-to-speech-into-python-application)

Integrate speech-to-speech into your Python app using the Hugging Face pipeline. Follow our guide for a four-stage process: VAD, STT, LLM, and TTS for seamless voice interaction.

- Tags: how-to-guide
- Published: 2026-07-07

### [Python APIs for Speech-to-Speech: Building Low-Latency Pipelines with Hugging Face](/huggingface/speech-to-speech/python-apis-for-speech-to-speech)

Explore Python APIs for speech-to-speech with Hugging Face. Build low-latency VAD, STT, LLM, and TTS pipelines entirely in Python using the speech-to-speech repository.

- Tags: how-to-guide
- Published: 2026-07-07

### [How to Use the Speech-to-Speech Command-Line Interface: A Complete Guide](/huggingface/speech-to-speech/how-to-use-command-line-interface-speech-to-speech)

Master the speech-to-speech command-line interface with easy steps. Configure VAD, STT, LLM, and TTS models for seamless voice command execution. Get started now.

- Tags: how-to-guide
- Published: 2026-07-07

### [How to Fine-Tune a Pre-Trained Speech-to-Speech Model: A Complete Guide](/huggingface/speech-to-speech/how-to-fine-tune-pretrained-speech-to-speech-model)

Learn how to fine-tune a pre-trained speech-to-speech model using Hugging Face. Customize STT, LLM, or TTS components with custom checkpoints for powerful results.

- Tags: how-to-guide
- Published: 2026-07-07

### [Supported Output Audio Formats in Hugging Face Speech-to-Speech](/huggingface/speech-to-speech/supported-output-audio-formats-speech-to-speech)

Discover supported output audio formats in Hugging Face Speech-to-Speech. Learn about WAV and raw PCM outputs with 16 kHz 16-bit little-endian encoding for your audio projects.

- Tags: api-reference
- Published: 2026-07-07

### [How Does the Speech-to-Speech Model Generate Output? A Deep Dive into the Pipeline](/huggingface/speech-to-speech/how-speech-to-speech-model-generates-output)

Explore the speech-to-speech model pipeline: VAD, STT, LLM, processing, TTS, and streaming. Understand how audio input becomes spoken output.

- Tags: deep-dive
- Published: 2026-07-07

### [How to Set Up the Speech-to-Speech Project Locally: A Complete Guide](/huggingface/speech-to-speech/how-to-set-up-speech-to-speech-project-locally)

Set up speech-to-speech locally with ease. Install the PyPI package, configure your API key, and run the command to launch the WebSocket server on localhost. Follow our complete guide.

- Tags: how-to-guide
- Published: 2026-07-07

### [What Is the Hugging Face speech-to-speech Repository? A Complete Guide to the Modular Voice Agent Pipeline](/huggingface/speech-to-speech/what-is-huggingface-speech-to-speech-repository)

Explore the huggingface speech-to-speech repository, a modular pipeline for real-time voice agents. Connect VAD, STT, LLM, and TTS with OpenAI Realtime compatibility.

- Tags: tutorial
- Published: 2026-07-07