How to Configure the Fish-Speech API Server with Custom Parameters

Configure the Fish-Speech API server by passing command-line arguments to python -m tools.api_server, including --listen, --device, --half, --llama-checkpoint-path, and --api-key, which are parsed in tools/server/api_utils.py and consumed by the API class at startup.

The Fish-Speech API server from the fishaudio/fish-speech repository is a lightweight ASGI service built with Kui and uvicorn. All runtime configuration options are centralized through the parse_args() function in tools/server/api_utils.py, allowing you to customize model checkpoints, hardware acceleration, authentication, and network binding via command-line flags when launching the server.

Available Configuration Parameters

The following runtime options are defined in tools/server/api_utils.py between lines 21 and 42. These values populate the application state during initialization in tools/api_server.py.

• --mode – Operation mode (currently supports only tts). Default: tts.
• --llama-checkpoint-path – Filesystem path to the LLaMA checkpoint used for semantic token generation. Default: checkpoints/s2-pro.
• --decoder-checkpoint-path – Path to the DAC decoder checkpoint file. Default: checkpoints/s2-pro/codec.pth.
• --decoder-config-name – Decoder configuration identifier (e.g., modded_dac_vq). Default: modded_dac_vq.
• --device – Preferred compute device (cuda, cpu, etc.). Default: cuda.
• --half – Load models in float16 precision to reduce VRAM usage. Disabled by default.
• --compile – Enable torch.compile for potential inference speed-ups. Disabled by default.
• --max-text-length – Upper character limit for incoming TTS requests (0 disables the limit). Default: 0.
• --listen – Host and port for the server to bind to. Default: 127.0.0.1:8080.
• --workers – Number of uvicorn worker processes to spawn. Default: 1.
• --api-key – Optional bearer token required for all API requests. Default: None.

Configuration Architecture Flow

Understanding how parameters flow from the command line to the inference engine ensures effective troubleshooting. The system processes configuration through four distinct layers:

1. Argument Parsing

The parse_args() function in tools/server/api_utils.py instantiates an argparse.ArgumentParser and returns a namespace object containing all validated command-line values. This occurs when the server boots in tools/api_server.py at lines 30-33.

2. API Class Initialization

The API class constructor in tools/api_server.py stores the parsed namespace in self.args and injects critical parameters into the ASGI application state. As implemented in lines 73-78, this makes values like app.state.device and app.state.max_text_length accessible to route handlers throughout the application lifecycle.

3. Model Manager Setup

During the initialize_app startup event, the ModelManager class defined in tools/server/model_manager.py receives the device preference, checkpoint paths, and precision flags (lines 38-51). It handles automatic device detection and loads both the LLaMA queue and DAC decoder before instantiating the TTSInferenceEngine.

4. Request Handling

REST endpoints defined in tools/server/views.py retrieve the shared tts_inference_engine from app.state.model_manager and enforce runtime constraints such as max_text_length on incoming requests (lines 40-46). If --api-key was provided, the bearer-token middleware defined in tools/api_server.py (lines 33-38) validates the Authorization header before allowing access to these routes.

Running the Server with Custom Settings

Execute the server using the tools.api_server module and append your desired configuration flags. Inspect all available options at any time by running python -m tools.api_server --help.

Launch with default settings:

python -m tools.api_server

Override the network binding, device, and enable half-precision:

python -m tools.api_server \
    --listen "0.0.0.0:9000" \
    --device cuda \
    --half

Use custom model checkpoints with text length limits and API key authentication:

python -m tools.api_server \
    --llama-checkpoint-path "checkpoints/custom-llama" \
    --decoder-checkpoint-path "checkpoints/custom-decoder/codec.pth" \
    --decoder-config-name "my_decoder_cfg" \
    --max-text-length 200 \
    --api-key "my-secret-token"

Run with multiple workers for improved concurrency on multi-core machines:

python -m tools.api_server \
    --workers 2 \
    --listen "[::1]:8080"

Advanced Configuration Options

Several parameters trigger specific behaviors within the model loading pipeline that affect performance and compatibility.

Device Fallback Logic

Even when you explicitly request --device cuda, the ModelManager in tools/server/model_manager.py (lines 30-36) automatically detects hardware availability. It will switch to MPS on Apple Silicon if CUDA is unavailable, or fall back to CPU if no GPU acceleration is present.

Precision Control

The --half flag forces torch.half (float16) precision, significantly reducing GPU memory consumption. Without this flag, line 28 of tools/server/model_manager.py defaults to torch.bfloat16 precision, which offers a balance between numeric stability and memory efficiency on modern NVIDIA hardware.

Model Compilation

Adding --compile enables torch.compile optimization within the TTSInferenceEngine. This may provide latency improvements on PyTorch 2.0+ installations with compatible CUDA capabilities, though compile times will increase during the initial model load.

Authentication Middleware

When --api-key is specified, the server activates bearer-token validation middleware. Clients must include the HTTP header Authorization: Bearer <your-key> with every request. Requests missing or providing incorrect tokens receive a 401 Unauthorized response before reaching the inference endpoints.

Summary

• All configuration for the Fish-Speech API server occurs through command-line arguments parsed by parse_args() in tools/server/api_utils.py.
• Key parameters include --llama-checkpoint-path, --decoder-checkpoint-path, --device, --half, and --listen for customizing hardware, models, and network binding.
• The configuration flow moves from argument parsing in tools/api_server.py to state injection, then to ModelManager initialization, and finally to request handlers in tools/server/views.py.
• Security features like --api-key enable bearer-token authentication via middleware defined in tools/api_server.py.
• Hardware optimization flags such as --half and --compile control memory precision and JIT compilation within the inference engine.

Frequently Asked Questions

What file contains the argument definitions for the Fish-Speech API server?

The parse_args() function in tools/server/api_utils.py (lines 21-42) defines all available command-line arguments using Python's argparse module. This file is imported by tools/api_server.py to build the configuration namespace consumed by the API class.

How do I enable GPU memory optimization when configuring the API server?

Pass the --half flag when starting the server. This forces the ModelManager in tools/server/model_manager.py to load weights using torch.half (float16) precision instead of the default torch.bfloat16, reducing VRAM usage by approximately 50% with minimal impact on audio quality.

Can I run the Fish-Speech API server on CPU-only machines?

Yes. Specify --device cpu at launch. The ModelManager automatically handles device fallback logic, switching to CPU if CUDA is unavailable. Note that inference latency will be significantly higher compared to GPU acceleration, and you should avoid using --compile as it provides limited benefit on CPU-only PyTorch builds.

How do I secure the API server with authentication?

Set the --api-key parameter to a secret string when launching the server. This activates bearer-token middleware in tools/api_server.py (lines 33-38) that validates the Authorization: Bearer <key> header on every request. Clients must include this header exactly as specified, or the server will reject the request with a 401 status code before reaching inference endpoints.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}