# VoiceStudio Backend Performance Metrics: A Complete Technical Guide

> Explore VoiceStudio backend performance metrics like latency, RTF, and throughput. This technical guide details how to access these crucial FastAPI metrics via API, WebSocket, or logs.

- Repository: [Palash Debnath/VoiceStudio](https://github.com/debpalash/VoiceStudio)
- Tags: performance
- Published: 2026-09-11

---

**The VoiceStudio backend exposes six quantitative performance metrics—`latency_ms`, `rtf`, `gen_time_s`, `ttfa_ms`, throughput bytes/s, and operation-count budgets—generated by the FastAPI server and accessible via HTTP API, WebSocket streams, or server logs.**

The `debpalash/VoiceStudio` repository implements a text-to-speech (TTS) backend that instruments every stage of the synthesis pipeline to provide observability into computational efficiency. Understanding these VoiceStudio backend performance metrics allows developers to optimize inference speed, diagnose bottlenecks, and ensure real-time processing capabilities for streaming audio applications.

## Core Performance Metrics Explained

The backend captures distinct latency and throughput measurements that quantify both wall-clock responsiveness and computational efficiency.

### Latency and Real-Time Factor (RTF)

The **`latency_ms`** metric measures total wall-clock time from request reception to synthesis completion, including preprocessing overhead. According to the source code in [`backend/api/routers/tts_stream.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/api/routers/tts_stream.py), this is calculated by wrapping the synthesis step with `time.perf_counter()` timestamps (see lines 113 and 119).

The **`rtf`** (real-time factor) represents the ratio of generation time to audio duration (`gen_time_s / audio_duration_s`). Values below 1.0 indicate faster-than-real-time processing, meaning the system generates one second of audio in less than one second of compute time. This metric appears in the JSON response from the generation endpoint and within WebSocket stream messages.

### Generation Timing (`gen_time_s`)

The **`gen_time_s`** metric captures the raw CPU/GPU time spent executing TTS model forward passes, excluding network transfer overhead. In [`backend/api/routers/tts_stream.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/api/routers/tts_stream.py) (lines 131–138), the backend records the total time spent inside the generation pipeline, providing insight into model inference efficiency independent of I/O latency.

### Time-to-First-Audio (`ttfa_ms`)

For streaming applications, **`ttfa_ms`** (time-to-first-audio) measures the latency until the first audio chunk streams back to the client. The WebSocket endpoint at `/ws/tts` embeds this metric in stream messages (line 291 in [`backend/api/routers/tts_stream.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/api/routers/tts_stream.py)), enabling client-side progress indicators and quality-of-service monitoring.

### Throughput and Operation Budgets

The **throughput probe** measures bytes-per-second delivery capacity for multi-gigabyte downloads, implemented in [`backend/services/endpoint_race.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/services/endpoint_race.py) within the `throughput_probe` function. This metric resolves ambiguous latency verdicts when the backend selects between mirror endpoints.

**Operation-count budgets** enforce performance guards in [`tests/test_perf_operation_budgets.py`](https://github.com/debpalash/VoiceStudio/blob/main/tests/test_perf_operation_budgets.py), counting `generate` calls and `text-normalization` passes to guarantee that hot paths do not regress (e.g., ensuring streaming TTS calls the engine exactly once per chunk).

## Where Metrics Are Generated in the Source Code

Understanding the instrumentation points helps when extending or debugging the telemetry system.

- **[`backend/api/routers/tts_stream.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/api/routers/tts_stream.py)**: Records start/end timestamps using `time.perf_counter()` and returns `latency_ms`, `gen_time_s`, and `rtf` in the JSON payload.
- **[`backend/api/routers/ws/tts.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/api/routers/ws/tts.py)**: Emits `ttfa_ms` and per-chunk `rtf` values during WebSocket streaming synthesis.
- **[`backend/services/endpoint_race.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/services/endpoint_race.py)**: Contains the throughput probe logic that samples ranged-GET downloads to determine mirror selection based on measured download speed.
- **[`tests/test_perf_operation_budgets.py`](https://github.com/debpalash/VoiceStudio/blob/main/tests/test_perf_operation_budgets.py)**: CI test suite enforcing operation-count budgets to prevent hidden performance regressions.

## How to Retrieve Performance Metrics

The backend exposes metrics through both synchronous HTTP endpoints and asynchronous WebSocket connections.

### HTTP Generation Endpoint

```python
import requests

resp = requests.post(
    "http://127.0.0.1:3900/tts/generate",
    json={"text": "Hello world!", "engine": "omni"},
)
data = resp.json()
print(f"Latency: {data['latency_ms']} ms")
print(f"RTF: {data['rtf']:.2f}")

```

### WebSocket Streaming Metrics

```python
import websockets
import asyncio
import json

async def stream_tts():
    async with websockets.connect("ws://127.0.0.1:3900/ws/tts") as ws:
        await ws.send(json.dumps({"text": "Hello world!", "engine": "omni"}))
        async for msg in ws:
            payload = json.loads(msg)
            if "ttfa_ms" in payload:
                print(f"First audio after {payload['ttfa_ms']} ms")
            if "rtf" in payload:
                print(f"Chunk RTF: {payload['rtf']:.2f}")

asyncio.run(stream_tts())

```

### System Maintenance

When generations hit the 300-second timeout threshold, flush the model from memory:

```bash
curl -X POST "http://127.0.0.1:3900/system/flush-memory?unload_model=true"

```

## Summary

- **`latency_ms`**: Total wall-clock time from request to completion, measured in [`backend/api/routers/tts_stream.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/api/routers/tts_stream.py).
- **`rtf`**: Real-time factor indicating compute efficiency; values < 1.0 are faster than real-time.
- **`gen_time_s`**: Raw inference time excluding I/O, tracked in lines 131–138 of the TTS stream router.
- **`ttfa_ms`**: Streaming latency to first audio chunk, available via WebSocket at line 291.
- **Throughput probe**: Bytes-per-second measurement in [`endpoint_race.py`](https://github.com/debpalash/VoiceStudio/blob/main/endpoint_race.py) for mirror selection.
- **Operation budgets**: CI-enforced counters in [`test_perf_operation_budgets.py`](https://github.com/debpalash/VoiceStudio/blob/main/test_perf_operation_budgets.py) preventing regression.

## Frequently Asked Questions

### What is RTF in VoiceStudio?

**RTF** (real-time factor) is the ratio of generation time to audio duration calculated as `gen_time_s / audio_duration_s`. When the value is 0.5, the system generates audio twice as fast as real-time playback. The metric appears in both HTTP JSON responses and WebSocket streams from [`backend/api/routers/tts_stream.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/api/routers/tts_stream.py).

### How do I measure time-to-first-audio?

Connect to the WebSocket endpoint at `ws://127.0.0.1:3900/ws/tts` and inspect the `ttfa_ms` field in incoming messages. This value represents the milliseconds elapsed from request submission until the first audio chunk streams to the client, implemented at line 291 of the TTS stream router.

### Where are performance metrics logged?

Metrics are emitted through three channels: the HTTP API returns them in JSON payloads, WebSocket connections stream them as message fields, and the FastAPI server writes them to stdout logs. The authoritative documentation in [`docs/performance.md`](https://github.com/debpalash/VoiceStudio/blob/main/docs/performance.md) enumerates all environment variables affecting these measurements.

### How does VoiceStudio prevent performance regressions?

The repository includes [`tests/test_perf_operation_budgets.py`](https://github.com/debpalash/VoiceStudio/blob/main/tests/test_perf_operation_budgets.py), which enforces operation-count budgets in CI pipelines. This test suite counts `generate` calls and text-normalization passes to ensure hot paths maintain exact operation counts per chunk, preventing hidden efficiency regressions.