How to Run the Hugging Face Speech-to-Speech Pipeline in Offline Mode

Set HF_HUB_OFFLINE=1 and ensure all model assets are cached locally or loaded via explicit paths to run the Speech-to-Speech pipeline without network access.

The Hugging Face speech-to-speech repository provides a modular real-time voice conversation system that can operate entirely offline once its components are properly configured. This guide explains how to cache models, configure local backends, and eliminate all runtime network dependencies using environment variables and path overrides.

---

Understanding the Pipeline Architecture

The Speech-to-Speech pipeline assembles four independent stages at runtime: VAD → STT → LLM → TTS. Each stage uses a pluggable backend defined in the registry system.

In src/speech_to_speech/s2s_pipeline.py, the run_pipeline_command function (lines 167-187) orchestrates pipeline construction through ThreadManager and PipelineUnit classes. The _build_handlers method (lines 69-126) instantiates each backend by looking up specifications in src/speech_to_speech/backend_registry.py (lines 49-78), where STT_BACKENDS, LLM_BACKENDS, and TTS_BACKENDS map selector flags to concrete handler factories.

This modular design means offline operation is determined entirely by backend choice and asset availability—not by a monolithic configuration.

---

Step 1: Cache All Required Assets While Online

Before going offline, run your intended configuration once with network access. This downloads and caches all model weights, tokenizer files, Smart-Turn ONNX checkpoints, Silero-VAD data, and NLTK resources.

speech-to-speech serve \
    --stt parakeet-tdt \
    --llm_backend transformers \
    --model_name meta-llama/Meta-Llama-3-8B-Instruct \
    --tts qwen3 \
    --qwen3_tts_model_name Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice

Assets are stored in ~/.cache/huggingface/hub/ following the standard Hugging Face Hub cache structure. Verify completion by checking that no additional downloads occur on subsequent runs.

---

Step 2: Enable Offline Mode with Environment Variables

The Hugging Face Hub client respects the HF_HUB_OFFLINE environment variable. When set to 1, all HTTP requests are disabled and only local cache reads are permitted.

HF_HUB_OFFLINE=1 speech-to-speech serve \
    --stt parakeet-tdt \
    --llm_backend transformers \
    --model_name meta-llama/Meta-Llama-3-8B-Instruct \
    --tts qwen3

This guarantees zero network traffic. If any required file is missing from cache, the process fails fast with a clear error rather than attempting a download.

---

Step 3: Configure Local-Only LLM Backends

API-based LLM backends (responses-api, chat-completions) require network connectivity unless redirected to a self-hosted server. For fully offline operation, choose from these local backends registered in backend_registry.py:

Backend	Handler Class	Requirements
transformers	TransformersHandler	PyTorch, local GGUF or safetensors weights
mlx-lm	MLXLMHandler	Apple Silicon, MLX framework
llama-cpp	LlamaCppHandler	llama-cpp-python bindings

Alternatively, keep an API backend but point it locally:

# Terminal 1: Start local LLM server

llama-server -hf ggml-org/gemma-4-E4B-it-GGUF -np 2 -c 65536 -fa on --swa-full

# Terminal 2: Run pipeline with local API endpoint

HF_HUB_OFFLINE=1 speech-to-speech serve \
    --stt parakeet-tdt \
    --llm_backend responses-api \
    --responses_api_base_url http://127.0.0.1:8080/v1 \
    --responses_api_api_key "" \
    --tts qwen3

The responses_api_base_url override in src/speech_to_speech/arguments_classes/module_arguments.py Routes all LLM requests to your local server, bypassing external APIs entirely.

---

Step 4: Use Explicit Local Paths to Bypass Hub Lookup

Every backend accepts absolute filesystem paths through its --*_model_name argument. This skips the Hugging Face Hub identifier resolution and loads directly from disk.

speech-to-speech serve \
    --stt parakeet-tdt \
    --llm_backend transformers \
    --model_name /srv/models/Meta-Llama-3-8B-Instruct \
    --tts qwen3 \
    --qwen3_tts_model_name /srv/models/Qwen3-TTS-12Hz-1.7B-CustomVoice

Path overrides are processed in src/speech_to_speech/s2s_pipeline.py during handler instantiation. When a path exists locally, no cache lookup or hub API call occurs—even without HF_HUB_OFFLINE=1.

---

Handling the Smart-Turn Endpointing Model

The Smart-Turn component (implemented in src/speech_to_speech/VAD/smart_turn.py) uses an ONNX checkpoint for turn detection. Control its offline behavior with these options:

1. Pre-cache the checkpoint: Let it download once, then rely on HF_HUB_OFFLINE=1

2. Specify explicit path: Use --smart_turn_model_path /path/to/smart-turn-v3.2-cpu.onnx

3. Disable entirely: Add --no_smart_turn to skip the component

HF_HUB_OFFLINE=1 speech-to-speech serve \
    --stt parakeet-tdt \
    --llm_backend transformers \
    --tts qwen3 \
    --smart_turn_model_path /opt/models/smart-turn-v3.2-cpu.onnx

---

Complete Offline Deployment Checklist

• Run target configuration once online to populate ~/.cache/huggingface/hub/
• Verify STT model (e.g., parakeet-tdt) cached via huggingface-cli scan-cache
• Verify LLM weights cached or copied to explicit path
• Verify TTS model (e.g., qwen3) cached or copied to explicit path
• Verify Smart-Turn ONNX file available or disabled
• Export HF_HUB_OFFLINE=1 in environment or systemd unit
• Use local backends (transformers, mlx-lm, llama-cpp) or local API server
• Test startup with network interfaces disabled to confirm no timeouts

---

Summary

Running the Speech-to-Speech pipeline offline requires three conditions: all model assets must be locally available, the Hugging Face Hub client must be restricted with HF_HUB_OFFLINE=1, and backends must be configured to avoid remote API calls. The pipeline's registry-based architecture in backend_registry.py and flexible argument parsing in s2s_pipeline.py make this straightforward—cache once, then operate indefinitely without network connectivity.

• HF_HUB_OFFLINE=1 blocks all hub HTTP requests and forces cache-only operation
• Local backends (transformers, mlx-lm, llama-cpp, parakeet-tdt, qwen3) run inference on-device
• Path overrides bypass hub lookup entirely for custom model locations
• Smart-Turn can be pre-cached, path-specified, or disabled with --no_smart_turn

---

Frequently Asked Questions

What happens if a model file is missing when HF_HUB_OFFLINE=1?

The Hugging Face Hub client raises a LocalEntryNotFoundError immediately on startup. The error message specifies which file is missing and its expected cache location. You must either restore network access temporarily, manually copy the file to the cache path, or switch to an explicit local path that exists.

Can I mix online and offline backends in the same pipeline?

No—all components must have their assets available locally because HF_HUB_OFFLINE=1 is process-wide. However, you can use API-based backends like responses-api if the base_url points to a local server. The hub client remains offline while HTTP requests to 127.0.0.1 succeed.

How do I verify which files are cached before going offline?

Use the Hugging Face CLI: huggingface-cli scan-cache lists all downloaded models with their disk locations. For this pipeline specifically, check for directories matching your --model_name, --qwen3_tts_model_name, and Smart-Turn checkpoint identifiers. The cache follows the structure ~/.cache/huggingface/hub/models--{org}--{model}/snapshots/{commit}/.

Does offline mode affect audio I/O or WebSocket performance?

No. HF_HUB_OFFLINE=1 only disables Hugging Face Hub network operations. Audio capture, playback, and WebSocket communication in speech-to-speech serve operate normally. Performance characteristics depend solely on local hardware (GPU/CPU for inference, audio driver latency) and thread configuration in ThreadManager.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}