How to Implement Real-Time Streaming TTS with Fish-Speech

Fish-Speech ships with a production-ready HTTP API that streams synthesized WAV audio chunk-by-chunk as soon as the first samples are generated, eliminating wait time for full file generation.

Fish-Speech (fishaudio/fish-speech) is an open-source text-to-speech framework that includes native support for real-time streaming TTS through its built-in server. By leveraging an asynchronous generator pipeline, the system validates streaming requests, processes text through the inference engine, and delivers raw audio bytes progressively to minimize latency.

Architecture of the Streaming Pipeline

The streaming implementation consists of three tightly coupled components that bridge the synchronous TTS engine with an async HTTP interface.

The HTTP Endpoint (/v1/tts)

Located in tools/server/views.py (lines 28-64), the tts view function receives POST requests and validates the ServeTTSRequest payload. When req.streaming is set to true and the output format is "wav", the endpoint returns a StreamResponse whose iterable is the async generator inference_async(req, engine).

The Async Bridge (inference_async)

Found in tools/server/api_utils.py (lines 72-78), this helper function iterates over the synchronous generator returned by inference_wrapper and yields each chunk downstream. This design keeps the core TTS engine synchronous while exposing a non-blocking API to the ASGI server (Kui + Uvicorn).

The Inference Wrapper (inference_wrapper)

Implemented in tools/server/inference.py (lines 12-40), this wrapper drives the underlying TTSInferenceEngine and yields three types of byte chunks:

• Header chunk: Raw WAV header bytes yielded immediately
• Segment chunks: Audio data where float waveforms (result.audio[1]) are scaled by AMPLITUDE to 16-bit PCM and converted to bytes
• Final chunk: Last bytes signaling stream completion

Request Flow and Validation

To enable real-time streaming TTS, clients must satisfy specific constraints enforced by the server.

1. Request Requirements: The JSON (or MsgPack) body must include streaming: true and set format to "wav"—the only supported streaming format according to the validation logic in views.py.
2. Engine Integration: The ModelManager retrieves the TTSInferenceEngine from ASGI state, then the inference_wrapper invokes engine.inference(req) to generate audio segments.
3. Progressive Delivery: The ASGI server transmits each yielded bytes payload to the client immediately, achieving true real-time audio delivery without buffering the complete file.

Client Implementation Examples

Python Client with Requests

import requests

url = "http://localhost:8080/v1/tts"
payload = {
    "text": "Hello, real-time streaming!",
    "format": "wav",
    "streaming": True,
    "speaker": "default"
}

# stream=True enables raw chunk iteration

response = requests.post(url, json=payload, stream=True)

with open("output.wav", "wb") as f:
    for chunk in response.iter_content(chunk_size=4096):
        if chunk:
            f.write(chunk)

The server transmits the WAV header first, followed by audio segments as they are synthesized.

Testing with cURL

curl -X POST http://localhost:8080/v1/tts \
     -H "Content-Type: application/json" \
     -d '{"text":"Streaming test","format":"wav","streaming":true}' \
     --output streaming.wav

The --output flag captures the progressive stream directly to disk, creating a playable file as soon as the first chunk arrives.

Customization and Extension Points

You can modify the streaming behavior by editing specific source files:

• Add format support: Extend the validation in tools/server/views.py (lines 47-53) and update get_content_type in tools/server/api_utils.py to handle MP3 or FLAC streaming.
• Tune latency: Adjust the segment yield frequency in fish_speech/inference_engine.py to control how often the engine emits audio chunks.
• Authentication: The global api_auth middleware in api_server.py already protects routes; add custom logging inside the if req.streaming: block for audit trails.

Summary

• Fish-Speech implements real-time streaming TTS through the /v1/tts endpoint with streaming: true and WAV format.
• The pipeline uses inference_async in tools/server/api_utils.py to bridge sync engine output with async HTTP.
• inference_wrapper in tools/server/inference.py yields header, segment, and final byte chunks scaled to 16-bit PCM.
• Clients receive progressive audio streams with sub-second latency using standard HTTP streaming techniques.

Frequently Asked Questions

What audio format does Fish-Speech support for streaming?

Fish-Speech currently supports only WAV format for real-time streaming. The validation logic in tools/server/views.py explicitly checks that req.format == "wav" when streaming is enabled, as the chunk encoder assumes 16-bit PCM output.

How does the server handle backpressure during streaming?

The inference_async generator in tools/server/api_utils.py yields chunks as they become available from the synchronous inference_wrapper, while the ASGI server (Uvicorn) manages backpressure through its StreamResponse implementation, pausing the TTS engine if the client buffer fills up.

Can I stream to multiple clients simultaneously?

Yes. Because the streaming endpoint creates a new StreamResponse for each request and the TTSInferenceEngine handles inference requests independently, you can serve multiple concurrent streaming sessions. Each client receives its own generator chain through inference_async.

Where is the audio amplitude scaling handled?

The conversion from float waveform to 16-bit PCM occurs in tools/server/inference.py within inference_wrapper, where raw audio samples are multiplied by the AMPLITUDE constant before being packed into bytes for the stream.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}