# How to Configure Weighted Random Routing in Switchyard for A/B Testing

> Configure weighted random routing in Switchyard for A/B testing. Distribute traffic across models using TOML or Python API for efficient experimentation.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: how-to-guide
- Published: 2026-08-22

---

**Switchyard implements A/B testing through its random routing algorithm, allowing you to distribute traffic across target models by defining relative weights in a TOML configuration file or via the Python API.**

Weighted random routing in Switchyard enables controlled traffic distribution for A/B testing scenarios without external load balancers. This feature, part of the NVIDIA-NeMo/Switchyard repository, processes requests by selecting from multiple backend targets according to configurable probability distributions defined in your route manifest.

## Random Routing Algorithm Implementation

The core logic resides in [`crates/libsy/src/algorithms/rand.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/rand.rs), which implements the **random routing algorithm** using Rust's `WeightedIndex` distribution. When a request hits a route where `type = "random"`, Switchyard evaluates the **weights** array to calculate selection probabilities for each target in the **targets** list.

The algorithm supports **deterministic routing** through an optional `seed` parameter, ensuring reproducible target selection across server restarts—critical for debugging or controlled experiments.

## TOML Configuration for Weighted Routes

Define weighted random routes in your TOML manifest (e.g., [`dev-server/config.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/dev-server/config.toml)) using the following schema:

```toml
[routes.ab_test]
id      = "ab_test"
type    = "random"
targets = ["capable", "efficient"]
weights = [1, 3]
seed    = 42

```

**Key parameters:**
- **targets**: Array of model identifiers defined elsewhere in your configuration
- **weights**: Optional array of relative probabilities (defaults to 1 for all targets if omitted)
- **seed**: Optional integer for deterministic selection

### Weight Validation Rules

Switchyard validates weight arrays at server startup in [`crates/libsy/src/algorithms/rand.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/rand.rs). The system enforces three constraints:

- The **weights** array length must exactly match the **targets** array length
- All values must be **finite, non-negative** numbers
- At least one weight must be **greater than 0**

Violations raise a `SwitchyardError` immediately at startup, preventing invalid routing configurations from serving traffic.

## Python API for Dynamic Route Construction

The Python bindings in [`crates/switchyard-py/src/libsy_bindings.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-py/src/libsy_bindings.rs) expose the `random()` function through PyO3, wrapped by [`switchyard/libsy/algorithms.py`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/switchyard/libsy/algorithms.py). Construct routes programmatically when building dynamic configurations:

```python
from switchyard.libsy import algorithms

# Create weighted-random routing algorithm programmatically

ab_test_algo = algorithms.random(
    targets=["capable", "efficient"],
    weights=[1, 3],
    seed=42
)

```

This returns a routing algorithm instance that can be attached to Switchyard clients or passed to server configuration builders. The validation logic mirrors the TOML path, ensuring consistency across configuration methods.

## CLI Usage and Client Integration

Once configured, clients invoke weighted routes via the `--model` flag:

```bash
switchyard run --model ab_test --prompt "Explain quantum entanglement in simple terms."

```

Switchyard selects the target according to the supplied weights (in the example above, "efficient" receives 75% of traffic while "capable" receives 25%) and forwards the request to the selected backend.

## Summary

- **Weighted random routing** in Switchyard uses relative weights (not percentages) to distribute traffic across multiple targets
- Configuration occurs in TOML files through the `[routes.<name>]` block with `type = "random"`
- The implementation resides in [`crates/libsy/src/algorithms/rand.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/rand.rs) using `WeightedIndex` for selection
- Validation occurs at server startup, requiring matching array lengths and at least one positive weight
- Python API access is available through `switchyard.libsy.algorithms.random()`
- Optional **seed** parameters enable deterministic, reproducible routing decisions

## Frequently Asked Questions

### What is the difference between relative weights and percentages in Switchyard?

Switchyard interprets weights as **relative** probabilities rather than strict percentages. A configuration with `weights = [1, 3]` assigns 25% traffic to the first target and 75% to the second, but Switchyard calculates this proportionally rather than requiring values that sum to 100. If weights are omitted, every target receives an implicit weight of 1, resulting in uniform random distribution.

### How does the seed parameter enable deterministic routing?

Providing a `seed` integer initializes the random number generator with a fixed state, causing Switchyard to select the same sequence of targets for identical request sequences across server restarts. This reproducibility, implemented in [`crates/libsy/src/algorithms/rand.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/rand.rs), allows developers to debug routing behavior or maintain consistent experimental conditions during A/B testing.

### What happens if I provide invalid weights in my TOML configuration?

Switchyard validates weight arrays immediately at server startup. If you provide negative numbers, non-finite values, a mismatched array length, or an array where all weights equal zero, the system raises a `SwitchyardError` and prevents the server from starting. This validation occurs in the Rust layer before any traffic processing begins, ensuring no invalid routing states reach production.

### Can I use weighted random routing with more than two targets for multivariate testing?

Yes. The **targets** and **weights** arrays support any number of backend models. Simply extend both arrays equally—`targets = ["model_a", "model_b", "model_c"]` with `weights = [2, 1, 1]` sends 50% traffic to model_a and 25% to each of the others. The `WeightedIndex` implementation in [`crates/libsy/src/algorithms/rand.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/rand.rs) efficiently handles arbitrary target counts.