# How to Configure Weighted Random Routing for A/B Testing in Switchyard TOML Deployments

> Configure weighted random routing for A B testing in Switchyard TOML with weights array to split traffic proportionally across models for reproducible results.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: how-to-guide
- Published: 2026-08-15

---

**Assign a `weights` array parallel to your `targets` array in the `[routes.<name>]` section to split traffic proportionally across models for reproducible A/B testing.**

Switchyard's **random routing** algorithm enables traffic splitting across multiple model targets without prompt inspection. According to the NVIDIA-NeMo/Switchyard source code, this approach is ideal for benchmark comparisons, staged rollouts, and controlled experiments where you need precise control over request distribution.

## Understanding Weighted Random Routing in Switchyard

The random router selects a target independently for every request. When you provide **weights**, Switchyard normalizes them into probabilities. Omitting `weights` triggers equal distribution across all targets.

The routing logic lives in [`crates/libsy/src/algorithms/random.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/random.rs). At startup, the server validates that:
- Every weight is a finite, non‑negative number
- At least one weight is greater than zero

The optional **`seed`** parameter ensures deterministic selection order—critical for reproducible test suites.

## Required TOML Schema Elements

Configure weighted random routing under `[routes.<name>]` with these keys:

| Key | Required | Description |
|-----|----------|-------------|
| `type` | Yes | Must be `"random"` |
| `targets` | Yes | Ordered array of target names defined in `[targets.*]` sections |
| `weights` | No | Ordered array of relative weights matching `targets` order |
| `seed` | No | Integer seed for reproducible random selection |

The TOML schema is documented in [`docs/reference/toml_schema.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/reference/toml_schema.md) (lines 30–34) and the detailed usage guide appears in [`docs/routing_algorithms/random_routing.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/routing_algorithms/random_routing.md).

## Basic Configuration: 30/70 A/B Split

This example routes 30% of traffic to GPT-4o and 70% to GPT-4o-mini:

```toml
schema_version = 1

[llm_clients.openrouter]
format = "openai_chat"
base_url = "https://openrouter.ai/api/v1"
api_key_env = "OPENROUTER_API_KEY"

[targets.strong]
id = "openai/gpt-4o"
llm_client = "openrouter"

[targets.weak]
id = "openai/gpt-4o-mini"
llm_client = "openrouter"

[routes.ab_test]
id = "ab-test"
type = "random"
targets = ["strong", "weak"]
weights = [3, 7]          # 3:7 ratio → ~30% / 70%

seed = 42                 # reproducible selection order

```

The weights `3` and `7` are relative—Switchyard computes probabilities as `3/10 = 0.3` and `7/10 = 0.7`. Any proportional values work (e.g., `[30, 70]` or `[0.3, 0.7]`).

## Equal-Weight A/B Test: Omit Weights

For a 50/50 split, omit the `weights` key entirely:

```toml
[routes.ab_test]
id = "ab-test"
type = "random"
targets = ["strong", "weak"]

# No weights → equal 50/50 distribution

```

This matches the pattern found in [`dev-server/config.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/dev-server/config.toml) for the `[routes.random]` definition.

## Temporarily Disabling a Target

Set a target's weight to zero to stop routing traffic without removing the configuration:

```toml
[routes.ab_test]
id = "ab-test"
type = "random"
targets = ["strong", "weak"]
weights = [1, 0]          # 100% to "strong", 0% to "weak"

```

Useful for maintenance windows or emergency rollbacks.

## Validating and Running Your Deployment

Start the Switchyard server with your TOML configuration:

```bash
export OPENROUTER_API_KEY="your-openrouter-key"

# Validate configuration without starting services

switchyard-server --config routes.toml --dry-run

# Start the server

switchyard-server --config routes.toml --host 127.0.0.1 --port 4000

```

Send requests through the route ID (`ab-test` in these examples):

```bash
curl http://localhost:4000/v1/chat/completions \
     -H "Content-Type: application/json" \
     -d '{"model":"ab-test","messages":[{"role":"user","content":"Hello"}]}'

```

Each request triggers independent weighted selection. With `seed = 42`, the sequence is deterministic across server restarts.

## Key Source Files

| File | Purpose |
|------|---------|
| [`crates/libsy/src/algorithms/random.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/random.rs) | Rust implementation of weighted random selector |
| [`docs/routing_algorithms/random_routing.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/routing_algorithms/random_routing.md) | Usage guide with weighted examples and seed behavior |
| [`docs/reference/toml_schema.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/docs/reference/toml_schema.md) | Formal TOML schema for `[routes.<name>]` including `weights` |
| [`dev-server/config.toml`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/dev-server/config.toml) | Working example of random route configuration |
| [`examples/experimental/litellm/README.md`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/examples/experimental/litellm/README.md) | Integration with LiteLLM gateway (experimental) |

## Summary

- **`weights`** array controls proportional traffic split; omitted means equal weights
- **Validation** requires finite, non-negative values with at least one positive weight
- **`seed`** enables reproducible test sequences across restarts
- **Zero weights** effectively disable targets without configuration changes
- Core algorithm implemented in [`crates/libsy/src/algorithms/random.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/random.rs)

## Frequently Asked Questions

### What happens if weights don't sum to 1 or 100?

Switchyard normalizes weights automatically. Values `[3, 7]`, `[0.3, 0.7]`, and `[30, 70]` produce identical 30%/70% splits. The algorithm divides each weight by the total sum.

### Can I use weighted random routing with more than two targets?

Yes. The `targets` and `weights` arrays accept any matching length. Add additional target names and corresponding weights for multi-variant tests.

### Does the seed guarantee the same model answers the same request?

No. The seed makes the **selection sequence** deterministic, not request-to-model mapping. Two identical requests in sequence may still hit different models due to independent draws—use the seed for reproducible aggregate distributions in load tests.

### What validation errors occur with invalid weights?

Switchyard rejects configurations where any weight is non-finite (NaN, infinity) or negative, or where all weights are zero. Check server logs at startup for TOML validation failures.