# How to Configure Multiple Backend Targets for A/B Testing with Random Routing in Switchyard

> Learn to configure multiple backend targets for A/B testing with random routing in Switchyard. Define targets, set route type to random, and control traffic splitting with weights and seeds.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: how-to-guide
- Published: 2026-08-15

---

**To configure A/B testing with random routing in Switchyard, define multiple backend targets in your TOML configuration and create a route with `type = "random"` that specifies those targets along with optional weights and a seed for reproducible traffic splitting.**

Switchyard is an open‑source LLM routing framework developed by NVIDIA that enables dynamic traffic distribution across multiple inference backends. When you need to compare model performance, latency, or cost, random routing provides a straightforward mechanism to split requests between two or more targets without complex load‑balancing logic. Configuring this properly requires understanding how Switchyard bridges Python configuration with its Rust‑core routing algorithms.

## Understanding Switchyard's Random Routing Architecture

Switchyard implements random routing through a layered architecture that separates configuration parsing from algorithm execution. The routing decision happens in Rust for performance, while Python handles configuration and API exposure.

| Component | Source File | Responsibility |
|-----------|-------------|--------------|
| **Python algorithm binding** | [`switchyard/libsy/algorithms.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/algorithms.py) | Re‑exports the Rust `random` function for Python access |
| **Rust core implementation** | [`switchyard_rust/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/libsy.py) | Executes `random(targets, weights, seed) → RouteResult` |
| **TOML schema defaults** | `switchyard/cli/defaults/*.toml` | Defines valid `[targets]` and `[routes]` structures |
| **CLI entry point** | [`switchyard/cli/switchyard_cli.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/cli/switchyard_cli.py) | Loads configuration and exposes `--model` route selection |
| **Reference implementation** | [`examples/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/libsy.py) | Demonstrates programmatic random routing usage |

The `random` function in [`switchyard_rust/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/libsy.py) accepts three parameters:

- `targets: Vec<String>` — List of target identifiers to choose from
- `weights: Option<Vec<f64>>` — Optional probability distribution (must sum to 1.0)
- `seed: Option<u64>` — Optional seed for deterministic pseudo‑random selection

When `weights` is omitted, the algorithm applies uniform probability across all targets. The `seed` parameter is particularly valuable for A/B testing, ensuring identical request sequences trigger the same backend selection across test runs.

## Step‑by‑Step Configuration for A/B Testing

### Step 1: Declare Backend Targets

Each inference backend requires a dedicated `[targets.*]` section in your TOML configuration. Targets reference provider‑specific model identifiers and client configurations.

```toml
[targets.gpt4o]
id = "openai/gpt-4o"
llm_client = "openai"

[targets.claude_sonnet]
id = "anthropic/claude-3.5-sonnet"
llm_client = "openrouter"

```

The `id` field corresponds to the model name recognized by your LLM provider. The `llm_client` field must match a `[llm_clients.*]` block defined elsewhere in your configuration file, specifying authentication and connection parameters.

### Step 2: Configure the Random Route

Create a `[routes.*]` section with `type = "random"` to activate probabilistic target selection.

```toml
[routes.ab_test]
id = "ab-test"
type = "random"
targets = ["gpt4o", "claude_sonnet"]
weights = [0.7, 0.3]
seed = 12345

```

Critical configuration details:

- **`type = "random"`** — Required to invoke the random routing algorithm from [`switchyard/libsy/algorithms.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/algorithms.py)
- **`targets`** — Ordered list matching names from your `[targets.*]` declarations
- **`weights`** — Optional array defining traffic distribution; omit for equal 50/50 split
- **`seed`** — Optional unsigned 64‑bit integer guaranteeing reproducible selection sequences

When weights are specified, Switchyard validates that values sum to 1.0. The seed ensures that restarting your service produces identical routing decisions, essential for debugging A/B test discrepancies.

### Step 3: Launch with Route Selection

Reference your route by its `id` when starting Switchyard via the CLI.

```bash
switchyard launch claude --model ab-test

```

The `--model` flag accepts route identifiers, not target names. Switchyard's CLI parser in [`switchyard/cli/switchyard_cli.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/cli/switchyard_cli.py) loads the TOML configuration, validates route existence, and instantiates the specified algorithm with configured parameters.

### Step 4: Programmatic Access from Python

For applications embedding Switchyard directly, use the high‑level client or instantiate algorithms manually.

```python
import asyncio
from switchyard import launch

async def run_ab_test():
    # Launch client with random route selection

    client = launch("claude", model="ab-test")
    
    response = await client.chat(
        messages=[{"role": "user", "content": "Explain quantum computing"}]
    )
    print(f"Response from: {response.model}")  # Identifies which backend served request

    print(response.content)

asyncio.run(run_ab_test())

```

For direct algorithm access without the full client stack:

```python
from switchyard.libsy.algorithms import random

# Manually create algorithm instance

algo = random(
    targets=["gpt4o", "claude_sonnet"],
    weights=[0.7, 0.3],
    seed=12345
)

result = algo.select()  # Returns selected target name

print(f"Selected backend: {result.target}")

```

The `random` function imported from [`switchyard/libsy/algorithms.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/algorithms.py) wraps the Rust implementation, providing identical behavior to TOML‑configured routes.

## Validating Your Random Routing Configuration

Switchyard includes test coverage for random routing in [`tests/test_run_baseline_script.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/tests/test_run_baseline_script.py), which exercises configurations containing `[routes.random]` sections. Reference this file to understand expected configuration patterns and validation behavior.

Before production deployment, verify your setup:

1. **Check weight normalization** — Ensure `sum(weights) == 1.0` within floating‑point tolerance
2. **Test seed reproducibility** — Run identical request batches twice; target selection should match
3. **Validate target connectivity** — Confirm all declared backends respond to health checks
4. **Monitor selection distribution** — Log chosen targets over sufficient volume to confirm weight accuracy

## Extending to Multi‑Variant Testing

Random routing scales beyond binary A/B tests. Add additional targets and adjust weights for multi‑variant experiments:

```toml
[targets.mistral_large]
id = "mistralai/mistral-large"
llm_client = "openrouter"

[routes.multi_variant]
id = "multi-variant"
type = "random"
targets = ["gpt4o", "claude_sonnet", "mistral_large"]
weights = [0.5, 0.3, 0.2]

```

The [`switchyard_rust/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/libsy.py) implementation handles arbitrary target counts without configuration changes. Traffic splits can be adjusted dynamically by modifying weights and reloading configuration, enabling rapid experiment iteration.

## Summary

- **Switchyard A/B testing** relies on TOML configuration defining `[targets.*]` backends and `[routes.*]` with `type = "random"`
- The random algorithm lives in [`switchyard_rust/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/libsy.py) with Python bindings in [`switchyard/libsy/algorithms.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/algorithms.py)
- **Weights** enable non‑uniform traffic distribution; **seed** guarantees reproducible selection
- Launch routes via CLI `--model <route_id>` or programmatically through the `launch()` function
- Reference [`examples/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/libsy.py) and [`tests/test_run_baseline_script.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/tests/test_run_baseline_script.py) for working implementations

## Frequently Asked Questions

### Can I change traffic weights without restarting Switchyard?

Switchyard loads configuration at startup and does not currently support hot‑reload of routing parameters. To adjust weights, modify your TOML file and restart the service. For dynamic traffic shifting, consider deploying multiple Switchyard instances with different configurations behind an external load balancer.

### What happens if I omit the weights parameter in a random route?

The random algorithm defaults to uniform distribution across all specified targets. With two targets, this produces a 50/50 split; with three targets, 33.3/33.3/33.3, and so on. The [`switchyard_rust/libsy.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard_rust/libsy.py) implementation computes equal probabilities automatically when `weights: None` is passed.

### Does the seed parameter affect cryptographic security?

No. The seed controls deterministic pseudo‑random selection for testing reproducibility, not cryptographic operations. Switchyard's random routing uses standard PRNG mechanisms suitable for load distribution, not security‑sensitive applications. Do not rely on routing randomness for access control or encryption purposes.

### How do I troubleshoot which backend served a specific request?

Switchyard attaches target identification to response metadata. When using the Python client, inspect `response.model` or enable debug logging in [`switchyard/cli/switchyard_cli.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/cli/switchyard_cli.py) to emit routing decisions. For CLI usage, add `--verbose` flag to trace algorithm selection per‑request.