# How to Configure the Fish-Speech API Server with Custom Parameters

> Learn to configure the Fish-Speech API server with custom parameters like listen, device, and API key. Easily adjust your API setup for optimal performance.

- Repository: [Fish Audio/fish-speech](https://github.com/fishaudio/fish-speech)
- Tags: how-to-guide
- Published: 2026-03-12

---

**Configure the Fish-Speech API server by passing command-line arguments to `python -m tools.api_server`, including `--listen`, `--device`, `--half`, `--llama-checkpoint-path`, and `--api-key`, which are parsed in [`tools/server/api_utils.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/api_utils.py) and consumed by the `API` class at startup.**

The Fish-Speech API server from the `fishaudio/fish-speech` repository is a lightweight ASGI service built with **Kui** and **uvicorn**. All runtime configuration options are centralized through the `parse_args()` function in [`tools/server/api_utils.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/api_utils.py), allowing you to customize model checkpoints, hardware acceleration, authentication, and network binding via command-line flags when launching the server.

## Available Configuration Parameters

The following runtime options are defined in [`tools/server/api_utils.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/api_utils.py) between lines 21 and 42. These values populate the application state during initialization in [`tools/api_server.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/api_server.py).

- **`--mode`** – Operation mode (currently supports only `tts`). Default: `tts`.
- **`--llama-checkpoint-path`** – Filesystem path to the LLaMA checkpoint used for semantic token generation. Default: `checkpoints/s2-pro`.
- **`--decoder-checkpoint-path`** – Path to the DAC decoder checkpoint file. Default: `checkpoints/s2-pro/codec.pth`.
- **`--decoder-config-name`** – Decoder configuration identifier (e.g., `modded_dac_vq`). Default: `modded_dac_vq`.
- **`--device`** – Preferred compute device (`cuda`, `cpu`, etc.). Default: `cuda`.
- **`--half`** – Load models in **float16** precision to reduce VRAM usage. Disabled by default.
- **`--compile`** – Enable `torch.compile` for potential inference speed-ups. Disabled by default.
- **`--max-text-length`** – Upper character limit for incoming TTS requests (`0` disables the limit). Default: `0`.
- **`--listen`** – Host and port for the server to bind to. Default: `127.0.0.1:8080`.
- **`--workers`** – Number of uvicorn worker processes to spawn. Default: `1`.
- **`--api-key`** – Optional bearer token required for all API requests. Default: `None`.

## Configuration Architecture Flow

Understanding how parameters flow from the command line to the inference engine ensures effective troubleshooting. The system processes configuration through four distinct layers:

### 1. Argument Parsing

The `parse_args()` function in [`tools/server/api_utils.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/api_utils.py) instantiates an `argparse.ArgumentParser` and returns a namespace object containing all validated command-line values. This occurs when the server boots in [`tools/api_server.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/api_server.py) at lines 30-33.

### 2. API Class Initialization

The `API` class constructor in [`tools/api_server.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/api_server.py) stores the parsed namespace in `self.args` and injects critical parameters into the ASGI application state. As implemented in lines 73-78, this makes values like `app.state.device` and `app.state.max_text_length` accessible to route handlers throughout the application lifecycle.

### 3. Model Manager Setup

During the `initialize_app` startup event, the `ModelManager` class defined in [`tools/server/model_manager.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/model_manager.py) receives the device preference, checkpoint paths, and precision flags (lines 38-51). It handles automatic device detection and loads both the LLaMA queue and DAC decoder before instantiating the `TTSInferenceEngine`.

### 4. Request Handling

REST endpoints defined in [`tools/server/views.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/views.py) retrieve the shared `tts_inference_engine` from `app.state.model_manager` and enforce runtime constraints such as `max_text_length` on incoming requests (lines 40-46). If `--api-key` was provided, the bearer-token middleware defined in [`tools/api_server.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/api_server.py) (lines 33-38) validates the `Authorization` header before allowing access to these routes.

## Running the Server with Custom Settings

Execute the server using the `tools.api_server` module and append your desired configuration flags. Inspect all available options at any time by running `python -m tools.api_server --help`.

Launch with default settings:

```bash
python -m tools.api_server

```

Override the network binding, device, and enable half-precision:

```bash
python -m tools.api_server \
    --listen "0.0.0.0:9000" \
    --device cuda \
    --half

```

Use custom model checkpoints with text length limits and API key authentication:

```bash
python -m tools.api_server \
    --llama-checkpoint-path "checkpoints/custom-llama" \
    --decoder-checkpoint-path "checkpoints/custom-decoder/codec.pth" \
    --decoder-config-name "my_decoder_cfg" \
    --max-text-length 200 \
    --api-key "my-secret-token"

```

Run with multiple workers for improved concurrency on multi-core machines:

```bash
python -m tools.api_server \
    --workers 2 \
    --listen "[::1]:8080"

```

## Advanced Configuration Options

Several parameters trigger specific behaviors within the model loading pipeline that affect performance and compatibility.

### Device Fallback Logic

Even when you explicitly request `--device cuda`, the `ModelManager` in [`tools/server/model_manager.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/model_manager.py) (lines 30-36) automatically detects hardware availability. It will switch to **MPS** on Apple Silicon if CUDA is unavailable, or fall back to **CPU** if no GPU acceleration is present.

### Precision Control

The `--half` flag forces `torch.half` (float16) precision, significantly reducing GPU memory consumption. Without this flag, line 28 of [`tools/server/model_manager.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/model_manager.py) defaults to `torch.bfloat16` precision, which offers a balance between numeric stability and memory efficiency on modern NVIDIA hardware.

### Model Compilation

Adding `--compile` enables `torch.compile` optimization within the `TTSInferenceEngine`. This may provide latency improvements on PyTorch 2.0+ installations with compatible CUDA capabilities, though compile times will increase during the initial model load.

### Authentication Middleware

When `--api-key` is specified, the server activates bearer-token validation middleware. Clients must include the HTTP header `Authorization: Bearer <your-key>` with every request. Requests missing or providing incorrect tokens receive a 401 Unauthorized response before reaching the inference endpoints.

## Summary

- **All configuration** for the Fish-Speech API server occurs through command-line arguments parsed by `parse_args()` in [`tools/server/api_utils.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/api_utils.py).
- **Key parameters** include `--llama-checkpoint-path`, `--decoder-checkpoint-path`, `--device`, `--half`, and `--listen` for customizing hardware, models, and network binding.
- **The configuration flow** moves from argument parsing in [`tools/api_server.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/api_server.py) to state injection, then to `ModelManager` initialization, and finally to request handlers in [`tools/server/views.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/views.py).
- **Security features** like `--api-key` enable bearer-token authentication via middleware defined in [`tools/api_server.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/api_server.py).
- **Hardware optimization** flags such as `--half` and `--compile` control memory precision and JIT compilation within the inference engine.

## Frequently Asked Questions

### What file contains the argument definitions for the Fish-Speech API server?

The `parse_args()` function in **[`tools/server/api_utils.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/api_utils.py)** (lines 21-42) defines all available command-line arguments using Python's `argparse` module. This file is imported by **[`tools/api_server.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/api_server.py)** to build the configuration namespace consumed by the `API` class.

### How do I enable GPU memory optimization when configuring the API server?

Pass the **`--half`** flag when starting the server. This forces the `ModelManager` in [`tools/server/model_manager.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/server/model_manager.py) to load weights using `torch.half` (float16) precision instead of the default `torch.bfloat16`, reducing VRAM usage by approximately 50% with minimal impact on audio quality.

### Can I run the Fish-Speech API server on CPU-only machines?

Yes. Specify **`--device cpu`** at launch. The `ModelManager` automatically handles device fallback logic, switching to CPU if CUDA is unavailable. Note that inference latency will be significantly higher compared to GPU acceleration, and you should avoid using `--compile` as it provides limited benefit on CPU-only PyTorch builds.

### How do I secure the API server with authentication?

Set the **`--api-key`** parameter to a secret string when launching the server. This activates bearer-token middleware in [`tools/api_server.py`](https://github.com/fishaudio/fish-speech/blob/main/tools/api_server.py) (lines 33-38) that validates the `Authorization: Bearer <key>` header on every request. Clients must include this header exactly as specified, or the server will reject the request with a 401 status code before reaching inference endpoints.