Stage‑Router vs LLM‑Classifier Routing in Switchyard: Key Differences and When to Use Each

Stage‑Router uses tool‑result history and implicit signals to route requests, while LLM‑Classifier relies on a dedicated classifier model to judge each request's difficulty—two fundamentally different approaches to balancing quality versus cost in Switchyard.

Switchyard, NVIDIA's open‑source LLM routing framework, provides multiple strategies for directing requests between efficient (cheap/fast) and capable (high‑quality) model endpoints. The stage_router and llm_classifier strategies represent the two primary architectural patterns: signal‑based inference versus explicit classification. Understanding when to deploy each is critical for optimizing both performance and spend in production agent systems.

Primary Signal Sources: Implicit vs Explicit

The core difference lies in what data each strategy examines before making a routing decision.

Stage‑Router: Tool‑Result History as Primary Signal

Stage‑Router routing examines implicit signals derived from the agent's operational context:

• Error severity from recent tool executions
• Churn indicators (repeated attempts, rollbacks)
• Production intensity (frequency of file writes, test executions)
• Tool‑call patterns that reveal the agent's current "stage" (exploration, implementation, debugging)

These signals are aggregated into a composite confidence score between 0 and 1. The router only consults an optional LLM classifier when this score falls below the configured confidence_threshold.

LLM‑Classifier: Dedicated Model for Explicit Judgment

LLM‑Classifier routing bypasses implicit signals entirely. Instead, it sends each request (plus optional conversation context) to a separate classifier target—a designated LLM tasked solely with predicting whether the efficient model can handle the current turn.

The classifier returns a structured verdict (capable or efficient) with its own confidence value. No tool‑result history is required. This makes the strategy viable for pure chat‑completion workloads where agents do not invoke external tools.

Decision Flow Comparison

How Stage‑Router Makes Decisions

According to the Switchyard source code, Stage‑Router implements a tiered decision pipeline:

1. Compute confidence score from tool‑result signals
2. If score ≥ confidence_threshold: follow the signal's tier recommendation
3. If score < confidence_threshold: fall back to picker default or optional classifier
4. Apply critical‑error overrides routing to capable tier regardless of score

The decision_source field in observability headers reveals whether the choice originated from dimensions (signal‑based), llm‑classifier (fallback), override, or default.

How LLM‑Classifier Makes Decisions

The LLM‑Classifier strategy follows a simpler, model‑centric flow:

1. Send request + context to classifier.target LLM
2. Receive verdict (capable/efficient) and confidence
3. Route to efficient target only if verdict is efficient and confidence exceeds base_threshold
4. Otherwise, route to capable target

Optional threshold_step adjustments handle edge cases like unsupported verdicts or uncertain classifications.

Configuration in routes.toml

Both strategies are declared in routes.toml with distinct key sets.

Stage‑Router Configuration

[routes.stage]
id = "switchyard/stage"
type = "stage_router"
capable_target = "strong"
efficient_target = "weak"
picker = "efficient_first"
confidence_threshold = 0.5

# Optional: enable classifier fallback for low‑confidence turns

[routes.stage.classifier]
target = "strong"
prompt_template = "capability-classifier/prompt.md"

Full specification available in docs/routing_algorithms/stage_router_routing.md.

LLM‑Classifier Configuration

[routes.classifier]
id = "switchyard/llm_classifier"
type = "llm_classifier"
capable_target = "strong"
efficient_target = "weak"
classifier.target = "strong"    # LLM that judges capability

base_threshold = 0.6
threshold_step = 0.1

See docs/routing_algorithms/llm_classifier_routing.md for complete parameter reference. Example server configurations also appear in benchmark/server-configs/tb-lite-single-llm-classifier-*.toml.

When to Use Each Strategy

Scenario	Recommended Strategy	Rationale
Code‑writing agents with heavy tool usage	Stage‑Router	Rich signal surface from reads/writes/errors enables accurate stage inference without per‑turn LLM cost
Pure chat completions, no tool calls	LLM‑Classifier	No implicit signals available; explicit classification is the only viable signal source
Fine‑grained cost control needed	Stage‑Router	Avoids classifier LLM invocation on high‑confidence turns
Existing high‑quality classifier model	LLM‑Classifier	Leverages prior investment in task‑difficulty prediction
Fastest possible routing decisions	Stage‑Router	Signal‑based computation is cheaper than extra LLM call

Observability and Debugging

Both strategies emit identical response headers for monitoring:

• x-model-router-selected-model: Final target chosen
• x-model-router-rationale: Human‑readable explanation including decision source

For Stage‑Router, the rationale indicates whether the decision came from signals, override, classifier (fallback), or default. For LLM‑Classifier, it surfaces the classifier verdict and confidence.

Internal telemetry in switchyard/libsy/algorithms.py exposes decision_source values for downstream analytics on routing efficiency.

Launching Switchyard with Each Strategy

# Stage‑Router: efficient-first picker with signal-based routing

import subprocess
import os

os.environ["OPENROUTER_API_KEY"] = "sk-..."

subprocess.run([
    "switchyard", "launch", "claude",
    "--model", "stage",
    "--config", "routes.toml"
])

# LLM‑Classifier: dedicated classifier model per request

import subprocess
import os

os.environ["OPENROUTER_API_KEY"] = "sk-..."

subprocess.run([
    "switchyard", "launch", "claude",
    "--model", "llm_classifier",
    "--config", "routes.toml"
])

Summary

• Stage‑Router excels when agents produce rich tool‑result history; it infers operational stage from error severity, churn, and production intensity, with optional classifier fallback
• LLM‑Classifier is purpose‑built for workloads lacking implicit signals, using a dedicated classifier model to judge each request's difficulty
• Both strategies support efficient/capable tier splits, expose identical observability headers, and are configured via routes.toml with distinct parameter sets
• Default classifier prompts are defined in crates/libsy/src/prompts/capability-classifier/prompt.md and reused across both strategies when classification is required

Frequently Asked Questions

Can Stage‑Router operate without any classifier at all?

Yes. The [routes.<name>.classifier] block is optional. When omitted, low‑confidence turns fall back to the picker default (efficient_first or capable_first) rather than invoking an LLM classifier. This minimizes cost but sacrifices accuracy on ambiguous turns.

Does LLM‑Classifier support conversation context beyond the current request?

Yes. The classifier can optionally evaluate recent conversation turns alongside the current request. The specific context window is controlled by conversation management logic in switchyard/libsy/algorithms.py, not by the routing configuration itself.

What happens when the LLM‑Classifier returns an unsupported verdict?

The threshold_step parameter provides a confidence adjustment mechanism. If the classifier returns an unparseable verdict or confidence below operational thresholds, the step value can shift the effective threshold, typically biasing toward the capable tier to preserve output quality.

Are the same picker options available for both strategies?

No. picker (efficient_first/capable_first) is specific to Stage‑Router and governs fallback behavior when signals are insufficient. LLM‑Classifier has no picker; routing depends entirely on classifier output and threshold comparison.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}