# How to Debug the VoiceStudio Backend: FastAPI and gRPC Troubleshooting Guide

> Learn to debug the VoiceStudio backend efficiently. Troubleshoot FastAPI and gRPC issues by setting log levels, reloading the server, and using pdb for in-depth analysis.

- Repository: [Palash Debnath/VoiceStudio](https://github.com/debpalash/VoiceStudio)
- Tags: how-to-guide
- Published: 2026-09-11

---

**To debug the VoiceStudio backend, set `VOICESTUDIO_LOG_LEVEL=DEBUG`, run the FastAPI server with `--reload`, attach `pdb` to worker PIDs logged by the pool manager, and use the isolated test suite as a reproducible sandbox.**

VoiceStudio's backend is a FastAPI application that orchestrates a pool of gRPC worker processes to handle text-to-speech, translation, and subtitle segmentation tasks. Because the architecture splits logic between the FastAPI entry point and separate worker subprocesses, effective debugging requires understanding how to trace requests across process boundaries. This guide covers the essential source files and techniques you need to systematically debug the VoiceStudio backend according to the `debpalash/VoiceStudio` source code.

## Architecture Overview

Understanding the layer hierarchy helps you locate the exact file to instrument with breakpoints or logging.

### Entry Point and FastAPI Layer

The bootstrap process begins in [`backend/main.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/main.py), which initializes the FastAPI application, registers HTTP routes, and launches the gRPC server. When the application starts, it instantiates the worker pool and binds the transport layer to local ports.

### Worker Pool Management

The [`backend/worker/pool.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/pool.py) module contains the `WorkerPool` class that manages the lifecycle of worker processes—spawning, health checks, graceful shutdown, and automatic restarts. This file logs PID numbers when workers spawn, making it the primary reference for attaching external debuggers.

### gRPC Transport Layer

Communication between FastAPI and workers relies on a client-server gRPC pair:
- [`backend/worker/transport/server.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/transport/server.py) handles inbound RPC calls, TLS authentication, and request dispatch.
- [`backend/worker/transport/client.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/transport/client.py) wraps outgoing calls from FastAPI routes with retry logic and connection pooling.

### Service Implementations

Concrete business logic resides in `backend/services/`:
- **TTS**: [`backend/services/tts_backend.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/services/tts_backend.py) implements the text-to-speech pipeline.
- **Translation**: [`backend/services/translator.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/services/translator.py) handles language conversion.
- **Subprocess Fallback**: [`backend/services/subprocess_backend.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/services/subprocess_backend.py) provides a fallback for engines that cannot run in the primary process.

### TLS and Security

Mutual TLS for inter-process communication is configured in [`backend/worker/tls.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/tls.py). This module generates temporary certificates and validates peer identity; misconfigurations here manifest as cryptic handshake errors that require certificate regeneration.

## Enable Verbose Logging

The standard library `logging` module drives all diagnostic output. Set the environment variable `VOICESTUDIO_LOG_LEVEL=DEBUG` before starting the server to receive detailed traces from FastAPI, gRPC, and worker internals. Logs emit to stdout and include the `WorkerPool` spawn messages that list worker PIDs.

```bash
export VOICESTUDIO_LOG_LEVEL=DEBUG
uvicorn backend.main:app --log-level debug

```

## Run in Development Mode

For rapid iteration, launch the server with hot-reload enabled. This mode automatically restarts the process on code changes and displays full stack traces for unhandled exceptions.

```bash
uvicorn backend.main:app --reload --log-level debug

```

## Inspect and Attach to Worker Processes

Workers run as isolated subprocesses spawned by `WorkerPool` in [`backend/worker/pool.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/pool.py). To attach an interactive debugger:

1. Locate the worker PID in the logs (search for "Spawned worker PID").
2. Use `pdb` to attach to the running process.

```bash

# Find the worker PID

ps -ef | grep voice_studio_worker

# Attach Python debugger

python -m pdb -p <PID>

```

Alternatively, launch a worker manually in the foreground using the `if __name__ == "__main__"` block inside [`backend/worker/pool.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/pool.py) to bypass the multiprocessing layer during initial setup.

## Use the Test Suite as a Sandbox

The test files provide isolated environments that spin up full FastAPI and gRPC stacks without affecting production data. The file [`backend/tests/test_tts_backend_lifecycle.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/tests/test_tts_backend_lifecycle.py) demonstrates worker lifecycle management including restarts and crashes.

Run a single test with verbose output to reproduce specific failures:

```bash
pytest -vv backend/tests/test_tts_backend_lifecycle.py::test_restart_worker

```

Insert temporary `print()` or `logging` statements inside the test or the modules it exercises to inspect intermediate state changes without modifying production code paths.

## Debug TLS and gRPC Connectivity

TLS handshake failures usually indicate expired certificates or hostname mismatches. Regenerate certificates using the helper in [`backend/worker/tls.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/tls.py):

```bash
python -m backend.worker.tls generate

```

To validate gRPC connectivity manually, use the generated protocol stubs to call methods directly against a running worker:

```python
import grpc
from backend.worker.protocol.gen import worker_v1_pb2, worker_v1_pb2_grpc

channel = grpc.insecure_channel('localhost:50051')
stub = worker_v1_pb2_grpc.WorkerServiceStub(channel)

# Verify worker health

resp = stub.HealthCheck(worker_v1_pb2.HealthCheckRequest())
print('Worker health:', resp.status)

```

The client implementation in [`backend/worker/transport/client.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/transport/client.py) logs every RPC call at `DEBUG` level, including method names and `grpc.RpcError` details.

## Monitor Resource Limits and Crashes

The modules [`backend/worker/capacity.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/capacity.py) and [`backend/worker/deadlines.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/deadlines.py) enforce CPU, GPU, and timeout quotas. If requests abort with `CapacityExceeded` or `DeadlineExceeded`, increase the limits in [`backend/config.yaml`](https://github.com/debpalash/VoiceStudio/blob/main/backend/config.yaml) or tune the pool size in [`backend/worker/pool.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/pool.py).

When workers crash unexpectedly, the system captures stdout/stderr to `runtime/crash_reports/`. The test pattern in [`backend/tests/test_worker_crash_handling.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/tests/test_worker_crash_handling.py) demonstrates how to read these dumps, which contain stack traces and environment snapshots.

## Runtime Introspection Endpoint

For live diagnostics, query the debug endpoint exposed by [`backend/mcp_server.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/mcp_server.py):

```bash
curl http://localhost:8000/debug/info

```

This returns JSON with the current pool size, active worker PIDs, and recent error counts without requiring log file access.

## Summary

- **Set `VOICESTUDIO_LOG_LEVEL=DEBUG`** to expose granular logs from FastAPI and gRPC layers.
- **Use `--reload`** with `uvicorn` during development to catch exceptions immediately.
- **Attach `pdb` to PIDs** logged by [`backend/worker/pool.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/pool.py) to debug worker subprocesses.
- **Run isolated tests** via `pytest` against [`backend/tests/test_tts_backend_lifecycle.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/tests/test_tts_backend_lifecycle.py) to reproduce lifecycle bugs.
- **Regenerate TLS certificates** with [`backend/worker/tls.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/tls.py) when encountering handshake errors.
- **Query `/debug/info`** for real-time pool status and worker health metrics.

## Frequently Asked Questions

### How do I find the PID of a running VoiceStudio worker?

Check the application logs for messages emitted by `WorkerPool` in [`backend/worker/pool.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/pool.py) containing the phrase "Spawned worker PID". Alternatively, use `ps -ef | grep voice_studio_worker` to filter the process list for active worker processes.

### Why am I seeing TLS handshake errors between FastAPI and workers?

The mutual-TLS configuration in [`backend/worker/tls.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/tls.py) likely has expired or mismatched certificates. Run `python -m backend.worker.tls generate` to create fresh temporary certificates, or verify that the hostname in the certificate matches the address used by [`backend/worker/transport/client.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/transport/client.py).

### Can I debug a worker without running the full FastAPI server?

Yes. The [`backend/worker/pool.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/pool.py) file includes an `if __name__ == "__main__"` guard that allows you to execute a worker process in the foreground. This bypasses the multiprocessing spawn logic and lets you attach breakpoints before the gRPC server starts listening.

### What causes "CapacityExceeded" errors in the logs?

This error originates in [`backend/worker/capacity.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/capacity.py) when a request exceeds the configured CPU or GPU quotas defined in [`backend/config.yaml`](https://github.com/debpalash/VoiceStudio/blob/main/backend/config.yaml). Increase the resource limits in the configuration file, or scale the worker pool size in [`backend/worker/pool.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/pool.py) to distribute load across more processes.