# Passthrough vs Random Routing in Switchyard: A Complete Guide to LLM Request Routing Strategies

> Understand Switchyard's passthrough vs random routing strategies for LLM requests. Learn how to direct traffic efficiently to single or multiple models.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: deep-dive
- Published: 2026-08-15

---

**Passthrough routing forwards requests unchanged to a single upstream model, while random routing stochastically selects among multiple target models with optional weighting.**

Switchyard, NVIDIA's open-source LLM routing infrastructure, provides multiple **routing strategies** to control how requests are distributed across language models. Two fundamental approaches are the **passthrough** and **random** algorithms, each serving distinct operational needs. Understanding their differences is essential for building efficient, observable model serving pipelines.

## What Is Passthrough Routing in Switchyard?

Passthrough is the simplest routing strategy in Switchyard. It acts as a thin proxy layer that forwards incoming requests to exactly one upstream model without modification.

### Core Behavior

In [`crates/libsy/src/algorithms/passthrough.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/passthrough.rs), the implementation performs minimal work:

```rust
// From passthrough.rs - forwards request directly to target
tracing::info!(target = %self.target, "passthrough selected target");

```

The algorithm preserves the **upstream model identifier** rather than masking it behind a synthetic route name. This transparency makes passthrough ideal when downstream systems need to track specific model usage for billing or observability.

### Configuration Example

```toml
[routes.passthrough]
id = "switchyard/passthrough"
type = "passthrough"
targets = ["model/weak"]

```

The `targets` array must contain exactly one entry. The `type = "passthrough"` declaration identifies this route's behavior to the configuration parser in [`crates/switchyard-server/src/config.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/config.rs).

### Typical Use Cases

- **Simple proxying** to a single model provider
- **Preserving upstream model IDs** for external cost tracking
- **Minimal overhead** when routing logic is unnecessary

## What Is Random Routing in Switchyard?

Random routing introduces **stochastic request distribution** across multiple target models. Unlike passthrough, it uses a synthetic route identity while internally selecting a different upstream model for each request.

### Weighted Selection Algorithm

The implementation in [`crates/libsy/src/algorithms/rand.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/rand.rs) builds a `RandomClassifier` that supports weighted, stateless selection:

```rust
// From rand.rs - weighted random target selection
// "A `random` route that selects `first` before any request-local fallback"

```

The algorithm evaluates relative `weights` to determine selection probability. When weights are omitted, it defaults to uniform distribution across all targets.

### Configuration Example

```toml
[routes.random]
id = "switchyard/random"
type = "random"
targets = ["model/a", "model/b"]
weights = [0.7, 0.3]  # Optional: 70% to model/a, 30% to model/b

```

Multiple targets are required. The `weights` array, when provided, must match the length of `targets`.

### Typical Use Cases

- **Basic load balancing** across functionally equivalent models
- **Simple A/B testing** without dedicated experiment infrastructure
- **Stochastic fallback** when multiple viable providers exist
- **Cost optimization** by weighting cheaper models more heavily

## Key Differences Between Passthrough and Random Routing

| Dimension | Passthrough | Random |
|-----------|-------------|--------|
| **Target count** | Exactly one | Two or more |
| **Selection logic** | Deterministic (fixed) | Stochastic (weighted random) |
| **Model ID exposure** | Preserves upstream model ID | Masks behind synthetic route ID |
| **Configuration complexity** | Minimal | Moderate (requires targets list, optional weights) |
| **Performance overhead** | Negligible | Minimal (random number generation) |
| **Observability** | Upstream model visible in logs | Route logs selection decision |

## Using Both Strategies in Python

Switchyard's Python bindings in [`crates/switchyard-py/src/libsy_bindings.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-py/src/libsy_bindings.rs) expose both routing types:

```python
import switchyard

# Passthrough: direct forwarding to single model

direct_client = switchyard.Client(
    route="switchyard/passthrough",
    target="model/weak",
)

response = direct_client.chat_completion(
    messages=[{"role": "user", "content": "Explain quantum computing"}]
)

# Random: weighted distribution across models

balanced_client = switchyard.Client(
    route="switchyard/random",
    targets=["model/fast", "model/cheap", "model/accurate"],
    weights=[0.5, 0.3, 0.2]
)

response = balanced_client.chat_completion(
    messages=[{"role": "user", "content": "Summarize this article"}]
)

```

## Implementation and Testing

Both algorithms are thoroughly validated in [`crates/switchyard-server/tests/server.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/tests/server.rs):

- **Passthrough tests** verify single-target forwarding (lines 978–980)
- **Random routing tests** validate stochastic selection behavior (lines 702–704)

The metrics subsystem in [`crates/switchyard-server/src/metrics.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-server/src/metrics.rs) distinguishes between the two, noting that passthrough "does very little" while random routing emits per-target selection logs.

## Summary

- **Passthrough routing** provides transparent, single-model proxying with minimal complexity—use it when you need direct upstream model visibility and no distribution logic.

- **Random routing** enables weighted, stochastic load distribution across multiple models—use it for basic balancing, cost optimization, or simple experimentation without full A/B framework overhead.

- Both strategies are implemented in `crates/libsy/src/algorithms/` and configured via TOML in `crates/switchyard-server`, with Python bindings available for application integration.

## Frequently Asked Questions

### Can random routing work without weights?

Yes. If the `weights` field is omitted, Switchyard assigns equal probability to all targets. The `RandomClassifier` defaults to uniform distribution when no weight vector is provided.

### Does passthrough support multiple targets for fallback?

No. Passthrough is strictly single-target. For fallback behavior, use random routing with multiple targets or implement a custom routing algorithm.

### How does Switchyard log routing decisions?

Passthrough logs only the target confirmation at `INFO` level. Random routing logs the selected target per-request with the message pattern `"random routing selected target '{}'"`, enabling downstream audit trails.

### Can I combine passthrough and random routes in the same configuration?

Absolutely. Switchyard configurations can define multiple route entries with different types. A typical setup might use passthrough for stable production traffic and random routes for experimental or cost-sensitive workloads.