# How the Switchyard Stage Router Chooses Between Tool-Response Pattern Matching and an LLM Judge

> Discover how the Switchyard stage router selects between tool-response pattern matching and an LLM judge. Learn about its decision-making process for optimal routing.

- Repository: [NVIDIA-NeMo/Switchyard](https://github.com/NVIDIA-NeMo/Switchyard)
- Tags: how-to-guide
- Published: 2026-09-13

---

**The Switchyard stage router evaluates six built-in tool-response signals to calculate a confidence score; if that score exceeds a configurable threshold, it routes to the capable tier immediately, otherwise it either defaults to the efficient tier or consults an optional LLM classifier for a final verdict.**

The NVIDIA-NeMo/Switchyard framework provides an intelligent routing layer that dynamically selects between high-capability and high-efficiency language models on a per-turn basis. Understanding how the stage router balances deterministic signal processing against learned LLM judgment is essential for optimizing both cost and accuracy in production agent systems.

## Understanding the Two-Tier Routing Architecture

The stage router operates on a **capable** versus **efficient** model dichotomy. For every conversational turn, the system must decide whether complex reasoning (capable) or fast, cheap inference (efficient) is appropriate. Rather than relying on a single heuristic, the router implements a two-step decision mechanism that first analyzes tool execution patterns, then optionally defers to a neural judge when confidence is low.

## Signal-Based Routing with Tool-Response Pattern Matching

At the core of the stage router is a compact history analysis of recent tool results, including file reads, writes, edits, test runs, and shell commands. The system extracts six distinct signals from this telemetry to compute a routing confidence value.

### The Six Built-in Signals

| Signal | Description | Routing Effect |
|--------|-------------|----------------|
| **severity** | Windowed error severity from tool executions | Pushes toward **capable** |
| **spinning** | Repeated attempts without substantive reads/writes | Pushes toward **capable** |
| **exploring** | Planning or reading activity without output production | Pushes toward **capable** |
| **recent_production_intensity** | Frequency of writes/edits in the recent window | Pushes toward **efficient** |
| **confidence** | Derived `tanh`-squashed aggregate of signal weights | Determines threshold crossing |
| **override** | Hard-override flag for critical/fatal errors | Forces **capable** immediately |

According to the source code in [`crates/libsy/src/algorithms/util/tool_signals.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/util/tool_signals.rs), these signals are extracted from the conversation state and weighted to produce a composite score.

### Confidence Score Calculation

The router applies a **tanh squashing function** to the aggregated signal weights, yielding a normalized confidence value in the range **[0, 1]**. In [`crates/libsy/src/algorithms/util/stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/util/stage.rs), this value is exposed as the `CONFIDENCE_METRIC`. 

If the confidence score exceeds the user-defined `confidence_threshold` (defaulting to `0.5`), the router **overrides the picker’s default tier** and selects the opposite of the current default (e.g., flipping from efficient to capable). If the score falls below the threshold, the system respects the picker’s default strategy (`efficient_first` or `capable_first`) or proceeds to the optional LLM judge.

## When the LLM Judge Steps In

When tool-response patterns provide ambiguous evidence, the stage router can defer to an **LLM classifier** that acts as a learned judge to break ties.

### The Classifier Configuration

Enabling the LLM judge requires adding a `classifier` block to the route configuration in [`crates/libsy/src/algorithms/stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/stage.rs). When present, the router checks the signal-derived confidence score against the threshold. If the score is **below** the threshold, the turn is forwarded to the specified classifier route rather than to the default model tier.

```toml
[classifier]
id = "switchyard/llm_classifier"
type = "llm_classifier"
model = "openai/gpt-4o"

```

### The Decision Flow

The complete decision chain implemented in [`stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/stage.rs) follows this precedence:

1. Extract tool signals and compute confidence score
2. If confidence ≥ threshold: route based on signal override (capable or efficient)
3. If confidence < threshold and classifier configured: send to LLM judge
4. If confidence < threshold and no classifier: use picker default

This design minimizes expensive classifier calls by only invoking the LLM judge when historical tool signals are inconclusive.

## Practical Configuration Examples

### Example 1: Signal-Only Routing

To use the stage router with pure pattern matching and no LLM judge, omit the classifier block:

```toml
[schema]
schema_version = 1

[targets.strong]
id = "openai/gpt-4o"
llm_client = "openrouter"

[targets.weak]
id = "openai/gpt-4o-mini"
llm_client = "openrouter"

[routes.stage]
id = "switchyard/stage"
type = "stage_router"
capable_target = "strong"
efficient_target = "weak"
picker = "efficient_first"
confidence_threshold = 0.5

```

With this configuration, the router evaluates tool signals exclusively. High-severity errors or spinning behavior will trigger the capable model, while routine edits remain on the efficient tier.

### Example 2: Hybrid Routing with LLM Fallback

To enable the two-stage fallback where ambiguous signals trigger an LLM classifier:

```toml
[routes.stage]
id = "switchyard/stage"
type = "stage_router"
capable_target = "strong"
efficient_target = "weak"
picker = "efficient_first"
confidence_threshold = 0.5

[classifier]
id = "switchyard/llm_classifier"
type = "llm_classifier"
model = "openai/gpt-4o"

```

Here, any turn with a confidence score below `0.5` is passed to the classifier for a final capable/efficient determination before execution.

### Example 3: Python Invocation

The Python bindings exposed in [`crates/switchyard-py/src/libsy_bindings.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/switchyard-py/src/libsy_bindings.rs) allow direct invocation of the stage router:

```python
import switchyard
from switchyard import Runner

runner = Runner.from_toml("config.toml")
response = await runner.run(
    messages=[{"role": "user", "content": "Please refactor the database layer"}],
    route="stage"
)

```

The `Runner` automatically applies the signal scoring and classifier logic defined in the TOML configuration.

## Summary

- The **stage router** in NVIDIA-NeMo/Switchyard routes each conversational turn between capable and efficient model tiers.
- **Six tool-response signals** (severity, spinning, exploring, production intensity, derived confidence, and critical override) are aggregated using a tanh function to produce a [0, 1] confidence score in [`tool_signals.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/tool_signals.rs).
- A configurable **confidence threshold** (default `0.5`) determines whether the router overrides the default picker or proceeds to an optional LLM judge.
- The **LLM classifier** is only invoked when the signal confidence is below the threshold, reducing costs while maintaining accuracy on ambiguous inputs.
- Core implementation resides in [`crates/libsy/src/algorithms/stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/stage.rs) with metrics exported via [`crates/libsy/src/algorithms/util/stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/util/stage.rs).

## Frequently Asked Questions

### How does the Switchyard stage router calculate confidence scores?

The router extracts six signals from recent tool execution history defined in [`crates/libsy/src/algorithms/util/tool_signals.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/util/tool_signals.rs). These signals are weighted and passed through a **tanh squashing function** to produce a normalized confidence value between 0 and 1, as implemented in the scoring logic and exposed via the `CONFIDENCE_METRIC` in the utility module.

### When should I enable the LLM classifier versus relying on signals alone?

Enable the LLM classifier when your application encounters **ambiguous tool-response patterns** that deterministic heuristics cannot reliably classify, such as novel error types or nuanced planning phases. For well-defined workflows with clear success/failure signals (e.g., test suites or file writes), the signal-only approach reduces latency and cost by avoiding unnecessary classifier calls.

### What happens when the confidence score exactly equals the threshold?

When the confidence score equals the `confidence_threshold` exactly, the router treats this as meeting the threshold condition. According to the logic in [`crates/libsy/src/algorithms/stage.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/crates/libsy/src/algorithms/stage.rs), a score greater than or equal to the threshold triggers the signal-based override, routing to the tier indicated by the aggregated signals rather than falling back to the classifier or default picker.

### Can I customize the signals used by the stage router?

The current implementation provides a fixed vocabulary of six signals (severity, spinning, exploring, recent_production_intensity, confidence, and override) defined in [`tool_signals.rs`](https://github.com/NVIDIA-NeMo/Switchyard/blob/main/tool_signals.rs). While you cannot add custom signals without modifying the Rust source, you can tune the routing behavior by adjusting the `confidence_threshold` or by adding a **custom LLM classifier** that incorporates domain-specific logic for the ambiguous cases.