How Stage-Router Routing Uses Tool-Result Signals to Drive Agent-Based Routing in Switchyard

Stage-router routing in Switchyard decides whether to route each LLM turn to a capable (expensive) or efficient (cheap) model by analyzing tool-result history generated by coding agents, combining error severity, production intensity, and exploratory behavior into a confidence score that gates tier selection.

Stage-router is a production routing backend in NVIDIA's Switchyard framework that dynamically selects model tiers based on real-time signals from agent tool execution. Unlike static routing or pure LLM-based classification, it ingests structured feedback from coding agents—test results, file writes, error states—to make granular per-turn decisions. This article examines the signal extraction pipeline, confidence scoring mechanism, and decision flow as implemented in the Switchyard source.

Signal Extraction: The Two Axes of Tool-Result Analysis

The stage-router core logic lives in crates/switchyard-server/src/stats/algorithms/stage_router.rs and is documented in docs/routing_algorithms/stage_router_routing.md. After each tool call (search, run, test, edit), Switchyard appends the result to the conversation history. The router scores this history across two orthogonal dimensions:

• WRONG → capable: Negative signals that push routing toward the high-quality tier. These include error severity (exception types and stack traces), spinning (deep execution churn without observable reads or writes), and exploring (repeated file reads or planning without productive output).

• PROGRESS → efficient: Positive signals that justify staying on or switching to the cheaper tier. The primary metric is recent_production_intensity—measured as writes and edits within a configurable sliding window of recent turns.

Each raw signal is transformed into a signed score, squashed by a tanh function, and combined into a final confidence value bounded to [0, 1]. Per the documentation, a single strong signal yields approximately 0.46 confidence; a second corroborating signal pushes the composite past the default threshold of 0.5.

Confidence Thresholding and Tier Gating

The confidence_threshold parameter (default 0.5) acts as the primary gating mechanism. According to stage_router_routing.md lines 41-44:

┌─────────────────┐     ┌─────────────────────────┐     ┌─────────────────┐
│   Tool Result   │────▶│  Confidence ≥ 0.5?     │────▶│  Signal Picks   │
│    History      │     │                         │ yes │   Capable/Eff   │
└─────────────────┘     └─────────────────────────┘     └─────────────────┘
                                   │
                                   │ no
                                   ▼
                         ┌─────────────────────────┐
                         │   Classifier Configured? │
                         └─────────────────────────┘
                              │              │
                            yes              │ no
                              ▼              ▼
                    ┌─────────────┐    ┌─────────────┐
                    │ Classifier  │    │ Picker      │
                    │ Picks Tier  │    │ Default Tier│
                    └─────────────┘    └─────────────┘

If confidence meets or exceeds threshold, the router follows the signal's directional override. Below threshold, the decision cascades to an optional LLM classifier or falls back to the picker-configured default tier.

Picker Strategies: Efficient-First vs. Capable-First

The picker defines the baseline tier when signals are ambiguous or insufficiently confident. Switchyard supports two modes, documented at lines 66-73:

• efficient_first (recommended, default): The efficient tier is the baseline. Escalation to capable occurs only when WRONG signals generate decisive confidence. This optimizes for cost with quality guardrails.

• capable_first (experimental): The capable tier is the baseline. The turn demotes to efficient only after sustained PROGRESS signals with high confidence. Useful when quality cannot be compromised without explicit evidence of smooth progress.

Picker selection is configured per-route in the TOML specification.

Observability and Decision Provenance

Every routing decision exposes diagnostic metadata. Response headers include:

• x-model-router-selected-model: The chosen model identifier ("strong" or "weak" in typical configurations)
• x-model-router-rationale: Human-readable explanation (e.g., stage_router selected weak (confidence 0.612))

Internally, the router tracks decision_source as an enum with variants: override (critical error detected), tests_passed, dimensions (signal-based scoring), llm_classifier, or fall_open (picker default). This provenance enables operational debugging and cost attribution.

Configuration and Usage Examples

Defining a Stage-Router Route

Create routes.toml:

[routes.stage]
id = "switchyard/stage"
type = "stage_router"
capable_target = "strong"
efficient_target = "weak"
picker = "efficient_first"
confidence_threshold = 0.5
recent_turn_window = 3

Launch the server:

switchyard-server --config routes.toml --port 4000

Python Client Integration

import switchyard
from switchyard.launch import launch

launcher = launch(
    "claude",
    model="stage",
    config="routes.toml",
)

result = launcher.run("Refactor this recursive function to use dynamic programming")
print(result)

Inspecting Routing Decisions

for turn in launcher.session.turns:
    print(
        turn.headers["x-model-router-selected-model"],
        turn.headers["x-model-router-rationale"],
    )

Key Implementation Files

File	Purpose
docs/routing_algorithms/stage_router_routing.md	Design specification, signal definitions, picker behavior
crates/switchyard-server/src/stats/algorithms/stage_router.rs	Rust implementation of signal scoring and confidence calculation
switchyard/libsy/algorithms.py	Python façade for launcher integration
examples/libsy.py	Minimal library-usage demonstration
tests/test_launchers.py	Unit tests for tier selection logic

Summary

• Stage-router consumes tool-result history from coding agents to drive per-turn model selection
• Two signal axes—WRONG (escalate) and PROGRESS (maintain/reduce)—are scored and tanh-squashed into [0, 1] confidence
• The confidence_threshold (default 0.5) gates whether signals override the picker default
• Pickers (efficient_first, capable_first) define fallback tiers for ambiguous states
• Full observability via headers and decision_source enum enables operational transparency

Frequently Asked Questions

What signals indicate a model should escalate to the capable tier?

Three primary WRONG signals trigger escalation: error severity (unhandled exceptions, test failures), spinning (execution loops without filesystem activity), and exploring (repeated reads or planning without writes). These are detected from tool-result metadata in the conversation history and transformed into confidence scores via tanh squashing.

How does the recent_turn_window parameter affect routing?

The recent_turn_window (default 3 turns) bounds the lookback period for calculating recent_production_intensity—the count of writes and edits within recent history. A larger window smooths transient fluctuations but may delay detection of productivity drops. A smaller window reacts faster to changes but risks noise from single-turn anomalies.

Can stage-router operate without any LLM classifier?

Yes. The classifier is optional. If confidence_threshold is not met and no classifier is configured, the router falls back to the picker default tier (efficient_first or capable_first). This pure signal-driven mode reduces latency and eliminates classifier costs, trading some nuance for predictability.

How does efficient_first compare to cost-based routing strategies?

efficient_first with a moderate threshold (0.5) approximates a cost-optimizing strategy with quality circuit breakers: it minimizes expensive calls except when error or stagnation signals demand intervention. Unlike fixed budgets or simple round-robin, it adapts to task difficulty in real time. The experimental capable_first inverts this for quality-critical applications where cost overruns are acceptable but unnecessary waste should still be eliminated.

{
  "mcpServers": {
    "instagit": {
      "command": "npx",
      "args": ["-y", "instagit@latest"]
    }
  }
}