# How VoiceStudio Routes TTS and ASR Across CUDA, MPS, ROCm, and CPU: Engine Selection Explained

> Learn how VoiceStudio routes TTS and ASR across CUDA, MPS, ROCm, and CPU by detecting the best compute device and matching tasks to compatible engine capabilities. Optimize your AI workloads.

- Repository: [Palash Debnath/VoiceStudio](https://github.com/debpalash/VoiceStudio)
- Tags: internals
- Published: 2026-09-06

---

**VoiceStudio routes text-to-speech and automatic speech recognition workloads by first detecting the best available compute device, then matching tasks to workers that advertise compatible engine capabilities.**

VoiceStudio implements a two-layer routing strategy that separates **hardware detection** from **engine selection**. This design ensures that TTS and ASR jobs run on the most capable hardware with an appropriate inference backend. Below is a complete breakdown of how the system prioritizes devices, validates compatibility, and dispatches tasks to the right worker.

---

## Device Detection Hierarchy: CUDA, ROCm, Intel XPU, DirectML, MPS, and CPU

The foundation of VoiceStudio's routing logic is `services.model_manager.get_best_device()`, which probes the host and returns a standardized device string.

### Priority Order from Source

The detection logic follows a strict priority ranking (see [[`backend/services/model_manager.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/services/model_manager.py#L54-L56)](https://github.com/debpalash/VoiceStudio/blob/main/backend/services/model_manager.py#L54-L56)):

- **CUDA or ROCm** (NVIDIA/AMD discrete GPUs)
- **Intel XPU** (Intel GPU accelerators)
- **DirectML** (Windows machine learning)
- **Apple MPS** (Apple Silicon neural engine)
- **CPU** (fallback)

```python
from services.model_manager import get_best_device

device = get_best_device()
print(device)  # "cuda", "mps", "cpu", etc.

```

The function queries `core.device_caps.detect_host_caps()` for raw capability detection, then filters through the priority list to return the first available option.

---

## GPU Compatibility Validation and Fallback

VoiceStudio does not blindly trust GPU detection. The `check_device_compatibility()` helper validates that the detected GPU actually works with the installed PyTorch build.

### Forced CPU Fallback for Incompatible GPUs

When a CUDA-capable GPU is present but PyTorch lacks support for its compute capability, the system warns the user and returns `"cpu"` (see [[`backend/services/model_manager.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/services/model_manager.py#L70-L88)](https://github.com/debpalash/VoiceStudio/blob/main/backend/services/model_manager.py#L70-L88)):

```python

# Simplified excerpt from get_best_device()

if not check_device_compatibility("cuda"):
    import warnings
    warnings.warn("CUDA GPU detected but unsupported by PyTorch build; falling back to CPU")
    return "cpu"

```

Users can override this safety check by setting `OMNIVOICE_FORCE_CUDA=1` before startup.

### ROCm-Specific Environment Handling

For AMD ROCm hosts, `get_best_device()` automatically sets `HSA_OVERRIDE_GFX_VERSION` when required for compatibility with specific GPU generations (see [[`backend/services/model_manager.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/services/model_manager.py#L14-L20)](https://github.com/debpalash/VoiceStudio/blob/main/backend/services/model_manager.py#L14-L20)).

---

## Engine Selection Through Worker Capability Matching

Hardware selection is only the first half. VoiceStudio must also ensure the requested **engine** (e.g., `indextts`, `whisper`, `faster-whisper`) is available on the chosen device.

### Task Structure and Engine Field

Every task submitted to the scheduler carries an `engine` identifier:

```python
from backend.worker.scheduler import Task

task = Task(
    operation="asr",
    engine="faster-whisper",   # Must match a worker's advertised capability

    model_id="large-v3",
    audio_data=b"...",
)

```

### Worker Capability Advertisement

Workers register themselves with a list of `pb.ModelCapability` objects. Each capability specifies both the **engine** name and the **backend** (device type) it supports (see [[`tests/test_worker_registry.py`](https://github.com/debpalash/VoiceStudio/blob/main/tests/test_worker_registry.py#L314-L319)](https://github.com/debpalash/VoiceStudio/blob/main/tests/test_worker_registry.py#L314-L319)):

```python

# Example capability from test suite

capability = pb.ModelCapability(
    engine="indextts",
    backend="cuda",        # "cuda", "mps", "cpu", etc.

    model_id="index-tts-v1",
    batch_size=8,
)

```

### The Scheduler's Selection Algorithm

`Scheduler.select_worker()` implements the matching logic in two phases (see [[`backend/worker/scheduler.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/scheduler.py#L562-L567)](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/scheduler.py#L562-L567)):

1. **Filter eligible workers** — Must be idle and meet GPU/CPU requirements from task constraints
2. **Select from capable workers** — Must support the task's `engine` and `model_id` on the current device

```python

# Simplified logic from select_worker()

eligible = [w for w in workers if w.is_idle() and w.meets_gpu_requirement(task)]
if not eligible:
    capable = [w for w in workers if w.supports(task.engine, task.model_id, device)]
    if not capable:
        raise NoEligibleWorker(f"No worker for engine={task.engine} on {device}")
    worker = self._rank_select(capable)  # priority or least-loaded

```

If no capable worker exists, the scheduler raises `NoEligibleWorker` with a descriptive message (see [[`backend/worker/scheduler.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/scheduler.py#L68-L92)](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/scheduler.py#L68-L92)).

---

## TTS and ASR Backend Integration

Both TTS and ASR services consume the same device detection and scheduling infrastructure.

### Shared Device Resolution

`services.tts_backend` and `services.asr_backend` import `get_best_device()` to determine where to run inference. The resulting device string propagates through the task's `deadline_policy` and into the worker's `execution_device()` method (see [[`backend/worker/scheduler.py`](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/scheduler.py#L52-L55)](https://github.com/debpalash/VoiceStudio/blob/main/backend/worker/scheduler.py#L52-L55)):

- `"cuda"` → CUDA tensors, `torch.cuda`
- `"mps"` → Metal Performance Shaders
- `"cpu"` → Standard CPU inference

### Engine Context Enforcement

The TTS backend uses an `engine_in_use` context manager to ensure thread-safe engine selection during model loading and inference. This prevents cross-engine interference when multiple TTS engines are registered on the same worker.

---

## Complete Routing Example

```python
from services.model_manager import get_best_device
from backend.worker.scheduler import Scheduler, Task

# 1. Hardware detection (runs once at startup or per-request)

device = get_best_device()  # Returns "cuda" on NVIDIA, "mps" on Apple Silicon, etc.

# 2. Task creation with explicit engine

scheduler = Scheduler(worker_registry=registry, strategy="least-loaded")
tts_task = Task(
    operation="tts",
    engine="indextts",
    model_id="index-tts-v1",
    params={"text": "VoiceStudio routes efficiently across hardware targets"},
    gpu_preference="any",  # or "cuda", "mps", "none"

)

# 3. Scheduler matches to capable worker

try:
    assigned_worker = scheduler.select_worker(tts_task)
    result = assigned_worker.execute(tts_task, execution_device=device)
except NoEligibleWorker as e:
    # No worker registered with indextts engine on detected device

    print(f"Routing failed: {e}")

```

---

## Summary

- **`get_best_device()`** determines hardware priority: CUDA/ROCm → Intel XPU → DirectML → MPS → CPU, with GPU compatibility validation and CPU fallback
- **Worker capabilities** pair `engine` names with `backend` device types, enabling fine-grained routing
- **`Scheduler.select_worker()`** filters by eligibility, then matches task engine to worker capability on the detected device
- **TTS and ASR backends** share the same routing infrastructure, calling `get_best_device()` and respecting scheduler assignments
- **Fallback mechanisms** include forced CPU when GPU support is missing and environment variable injection for ROCm compatibility

---

## Frequently Asked Questions

### What happens if I have both CUDA and MPS available?

VoiceStudio prioritizes **CUDA/ROCm first**, then Intel XPU, DirectML, and finally **MPS**. Apple Silicon users with an external GPU must explicitly configure device preference if they want MPS over CUDA.

### Can I force a specific device even if VoiceStudio detects a better one?

Set environment variables before startup: `CUDA_VISIBLE_DEVICES=""` to hide CUDA, or `OMNIVOICE_FORCE_CUDA=1` to override compatibility checks. The scheduler also respects `gpu_preference="none"` in task parameters to force CPU.

### How does VoiceStudio handle engine versioning?

Workers advertise `model_id` alongside `engine` in their capabilities. The scheduler matches both fields, ensuring you don't route a `whisper-v2` task to a worker only capable of `faster-whisper-v3`.